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Abstract 

Productivity  in  the  United  States  is  declining.  We  have  slipped 
from  the  leader  of  the  world  to  eighth  among  nations.  Recognizing 
this.  Department  of  Defense  (DOD)  leaders  directed  the  implementation 
of  a  DOD  Productivity  Program.  The  key  to  any  productivity  program  is 
the  development  of  a  productivity  measurement  system.  The  purpose  of 
this  study  is  to  determine  a  managerial  method  for  productivity 
measurement  within  a  base  level  Civil  Engineering  Design  Section. 
Current  methods  of  productivity  measurement  were  reviewed  and  found  to 
be  deficient.  A  relatively  new  method,  Constrained  Facet  Analysis, 
was  chosen.  Design  section  inputs  and  outputs  were  determined.  A 
data  set  was  generated  to  a  predetermined  efficiency  result.  The 
Constrained  Facet  Analysis  model  was  run  using  the  generated  data,  and 
its  computer  output  was  compared  to  the  predetermined  efficiency 
result.  The  comparison  showed  identical  results  for  the  computer 
model  and  the  predetermined  efficiencies.  The  authors  concluded  that 
the  computerized  Constrained  Facet  Analysis  model  is  an  accurate  and 
valid  method  of  determining  productivity  within  a  base  level  Civil 
Engineering  Design  Section.  The  limitations  of  this  method  of 

productivity  measurement  are  discussed,  further  research,  using  data 
from  real  design  sections,  is  recommended  before  implementation  of  the 
model.  The  model  should  be  used  as  a  managerial  aid  to  improve 
organizational  performance.  It  should  not  be  used  to  evaluate 
manager's  performance. 
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DETERMINING  MANAGERIAL  METHODS  OF 
PRODUCTIVITY  MEASUREMENT  WITHIN 
CIVIL  ENGINEERING  DESIGN  UNITS 


I.  Introduction 


Background 

The  last  few  years  have  brought  the  issue  of  productivity  in 

American  economy  to  the  point  that  it  is  a  national  concern  (3:1). 

United  States  after  World  War  II  was  the  leader  in  the  economic  wo  J 

and  first  in  productivity.  That  trend,  however,  has  reversed  1 

productivity  is  now  in  a  downward  trend.  At  present,  the  United  Sta 

is  eighth  in  productivity  among  nations  (12).  American  iudustry  is  < 

longer  concerned  about  increased  productivity  or  creating  innovat 

ways  to  manufacture  products  more  efficiently  and  less  expensivel 

This  is  the  cause  for  the  decline  in  the  American  economy  over  the  1 

decade.  Nine  countries  now  have  a  greater  Gross  National  Product  r 

capita  than  does  the  United  States  (34).  For  many  years,  the  Uni  i 

States  was  consistently  number  one.  General  Alton  Slay  recognized  » 

problem  that  was  facing  the  United  States  when  he  said, 

My  conviction  is  now  very  strong  that  we  do  indeed  have  a 
national  problem... .*  national  industrial  productivity  disease 
which  must  be  addressed  if  we  are  to  maintain  our  status  as  t 
focus  of  the  free  world's  industrial,  economic,  and  military 
strength.  [16:2] 

The  American  economy  is  not  the  leader  it  once  was  fifteen  ye. 
ago.  This  decline  in  productivity  has  had  a  significant  impact  on 
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Air  Force  and  its  operations.  General  Louis  L.  Wilson,  Jr.,  summed  up 
this  impact  when  he  stated, 

The  Air  Force  is  facing  one  of  the  most  austere  times  in  its 
history.  In  spite  of  increased  defense  budgets,  our  buying 
power  has  eroded  with  the  net  result  that  we  have  to  do  more 
with  less.  To  meet  this  challenge,  we  need  to  fully  utilize 
our  most  costly  and  important  resource. . .people. . .by  instilling 
in  them  a  sense  of  urgency  about  their  important  role  in  the 
conduct  of  the  Nation's  critical  enterprise. . .national 
security. . .and  in  doing  so  we  must  increase  their  productivity. 
[3:3] 

The  key  to  tur  ling  the  declining  American  economy  around  is  to 
increase  productivity  in  America  (34).  The  importance  of  productivity 
improvement  can  not  be  overstated.  Without  a  productivity  increase,  the 
United  States'  standard  of  living  will  decline  (16:7).  General  James  P. 
Mullins,  in  an  address  to  the  students  and  faculity  at  the  Air  Force 
Institute  of  Technology,  concluded,  "We  can  not  afford  not  to  be 
productive;  we  can  no  longer  let  productivity  wane  (28).”  Rising  costs 
have  squeezed  the  public's  resources  between  public  demands  for  service 
and  the  cost  of  supplying  those  services.  There  is  growing  resistance 
on  the  part  of  the  public  to  pay  for  these  services  through  higher 
taxes.  This  creates  a  problem.  The  answer  to  this  problem  is  to  improve 
productivity  of  the  public  service  (18:9).  Major  General  Robert  C. 
Thompson,  former  director  of  USAF  Civil  Engineering,  stated  that  a  look. 
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into  the  future  indicates  that  the  Civil  Engineer  must  do  more  with  less 
through  increased  productivity  (3:4).  The  key  to  the  future  of  America 
was  summed  up  by  Thomas  Edison  when  he  said,  "  There  is  a  way  to  do  it 
better ...  find  it  (12).” 

On  October  23,  1978,  a  memo  from  President  Carter  was  sent  to  the 
head  of  all  Federal  departments  and  agencies.  The  memo  announced  the 


S: 

IV. 

w. 


.  > 


establishment  of  the  National  Productivity  Council.  The  council  is  to 

be  the  focal  point  for  productivity  improvement  efforts  in  the  United 

States  (36:1).  President  Carter  wrote  the  following  in  the  memo: 

I  have  established  this  council  in  recognition  of  the  vital 
role  productivity  plays  in  the  nation's  economy  by  helping 
control  inflation,  making  the  U.S.  goods  more  competitive  in 
world  markets,  and  increasing  the  real  income  of  the  American 
worker .. improved  productivity  is  vital  to  the  social  and 
economic  well-being  of  our  nation.  The  Federal  Government  can 
make  a  major  contribution  to  improving  productivity.  I  expect 
all  agencies  to  cooperated  and  assist  the  council  in  meeting 
its  responsibilites  so  we  realize  maximum  benefit  from  the 
Federal  effort  to  improve  productivity  growth.  [36:1] 


This  memo  was  followed  in  1981  by  the  Government  Cost  Reduction  Act 

(16:19).  This  act  was  an  attempt  by  the  federal  government  at 

productivity  enhancement. 

The  Secretary  of  Defense,  realizing  the  need  for  productivity,  made 

♦ 

increased  productivity  one  of  the  highest  priorities  of  the  Department 

of  Defense  (18:3)  The  Department  of  Defense  (DOD) ,  the  largest  and  most 

costly  of  all  public  organizations,  issued  directive  5010.31  that 

requires  each  military  department  to  estallish  a  productivity 

improvement  program  (36:1).  This  directive  was  followed  by  DOD 

Instruction  5010.34.  Howell  and  Van  Sickle,  in  a  master's  thesis, 

explained  the  meaning  of  the  Productivity  Program: 

DoD  Directive  5010.31  establishes  the  policy  of  focusing 
management  attention  on  the  achievement  of  maximum  defense 
output  within  available  resource  levels  by  . . .  seeking  and 
exploiting  opportunities  for  improved  methods  of  operations  in 
consonance  with  the  defense  preparedness  mission.  The  directive 
further  states  that,  productivity  measurement,  enhancement  and 
evaluation  will  be  an  integral  element  of  resource 
management. . .The  Directive  prescribes  a  labor-oriented  mode, 
but  allows  for  total  product  or  unit  cost  measures  if 
available .  [ 18  : 6 ] 
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In  response  to  the  White  House  and  the  DOD,  the  Air  Force  created  a 
comprehensive  productivity  improvement  plan.  This  plan  directed  all 
major  command  agencies  to  develop  individual  productivity  plans,  to 
appoint  "productivity  principals"  as  points  of  contact  for  productivity 
matters,  and  to  report  all  productivity  accomplishements  annually  to  Air 
Force  headquarters  (37:1).  In  issuing  the  plan,  General  James  A.  Hill, 
Vice  Chief  of  Staff,  said: 

Productivity  has  received  increased  emphasis  at  all  levels  of 
government,  and  had  consistently  surfaced  a  key  factor  during 
congressional  debates  on  Military  Appropriations.  If  we  are  to 
continue  obtaining  the  necessary  funding  for  vital  Air  Force 
Programs  everyone  at  all  organizational  levels  must  actively 
seek  more  productive  means  of  accomplishing  their  jobs.  We 
therefore  urge  your  full  support  for  this  plan.  [36:1] 

Air  Force  Civil  Engineering  has  long  felt  that  productivity 
improvement  was  important..  The  Base  Civil  Engineering  organization  is 
generally  the  largest  service  organization  on  base  and  usually  spends  40 
to  60  percent  of  the  total  operations  and  maintenance  budget  of  the  base 
(10:1).  Any  improvements  within  Civil  Engineering  would  have  a 
significant  impact  on  the  Air  Force  and  DOD  in  general.  The  Inefficient 
use  of  resources  by  USAF  Base  Civil  Engineering  organizations  has  a 
substantial  impact  on  the  overall  DOD  productivity  level  (3:3). 
Brigadier  General  Archie  S.  Mayes,  former  Deputy  Chief  of  Staff  for 
Civil  Engineering,  Strategic  Air  Command,  presented  a  plan  involving  six 
points  designed  to  improve  productivity  of  the  work  force  and  overall 
efficiency  of  Base  Civil  Engineering  operations  (3:4).  Major  General 
Guy  H.  Goddard,  also  a  former  Director  of  Civil  Engineering,  stressed 
that  the  key  to  productivity  within  Air  Force  Civil  Engineering  was  at 
the  base  lev^l  (3:4). 
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The  Air  Force  Civil  Engineering  community,  in  an  effort  to  increase 
productivity  instituted  an  automated  management  information  system 
called  the  Base  Engineer  Automated  Management  System  (BEAMS)  and  a 
Management  Review  Program  (MRP).  This  system  included  a  list  of 
objectives  to  be  used  as  aids  to  increasing  efficiency  (2:1-2).  The 
programs  are  intended  to  help  manage  the  civil  engineering  organization 
more  efficiently.  Presently,  the  civil  engineering  organizations  rely 
on  information  provided  by  the  BEAMS  system  to  monitor  performance.  The 
system  has  been  found  to  have  numerous  shortcomings.  In  a  move  to 
increase  productivity,  the  civil  engineering  community  is  attempting  to 
implement  a  new  automated  management  information  system  called  Work 
Information  Management  System  (WIMS).  Major  General  Clifton  D.  Wright, 
current  Director  of  Engineering  and  Services,  feels  so  strongly  about 
productivity  that  increasing  productivity  is  one  of  his  six  strategic 
goals.  Changes  to  AFR  85-1,  the  Civil  Engineering  Resources  and  Work 
Force  Management  Regulation,  have  come  about  as  management  efforts  aimed 
at  reducing  impediments  to  productivity  within  civil  engineering 
organizations  (3:5). 

Programs  have  been  implemented  on  a  national,  Air  Force,  and  Civil 
Engineering  level  all  designed  toward  increasing  productivity.  By 
increasing  productivity,  we  can  stop  rising  costs  that  are  squeezing  the 
public's  resources.  This  turn  can  only  be  achieved  if  all  levels  of 
industry  and  public  service  are  involved. 
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Justification  For  Study 

With  the  problem  of  declining  productivity,  the  manager  of  today  is 
faced  with  the  problem  of  what  to  do.  The  major  problem  is  that  a 
manager  can  not  improve  productivity  if  he  does  not  know  his  present 
productivity.  The  development  of  meaningful  productivity  measures  will 
ultimately  lead  to  better  management  in  the  areas  of  present  and  future 
operations  (20:13).  One  way  of  enhancing  and  improving  effective 
management  is  the  development  and  use  of  productivity  measures  (20:3). 

A  manager  is  given  a  certain  amount  of  input,  or  resources  which  he  uses 
to  produce  output.  After  these  outputs  are  produced,  performance 
feedback  information  and  measurement  is  needed  to  indicate  how  well  he 
utilized  his  resources  to  produce  the  output  (32:2). 

John  Mee  broke  the  management  process  down  into  seven  sub-processes 
(15:2).  Mee's  sixth  sub-process  deals  with  productivity  measurement. 
Not  only  are  measurements  of  productivity  beneficial  to  managers  but 
they  are  a  portion  of  managerial  responsibility.  Through  the  use  of 
Productivity  Measurement,  managers  can  be  helped  in  the  following  areas: 

1.  Current  Operations 

a.  To  objectively  identify  efficient  management 

b.  To  identify  and  take  effective  timely  remedial 
action  in  potential  trouble  areas 

c.  To  compare  the  relative  production  efficiency 
of  similar  functions  performed  in  different 
major  commands,  and 

d.  To  improve  productivity  and  the  methods  and 
standards  of  operation. 

2.  Aid  in  Future  Planning 

a.  To  improve  the  planned  allocation  of  resources 
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b.  To  improve  the  evaluation  of  effects  of  policy 
constraints  by: 

1)  Evaluating  feasibility 

2)  Making  more  effective  adjustment  to 
comply  with  externally  imposed 
constraint' 

3)  Measuring  advantages/disadvantages 
(costs)  of  externally  imposed  constraints 

c.  To  improve  the  integration  of  present 
policies  with  contingency  and  mobilization 
requirements.  [26:24] 


Kaneda  and  Wallett,  in  a  thesis  on  productivity,  concluded  that 
productivity  measurements  are  powerful  tools  for  any  manager.  But  these 
tools  are  more  difficult  to  develop  within  military  organizations 
because  of  substantive  goals  and  policy  constraints.  However,  some  type 
of  productivity  measurement  is  essential  to  assist  management  (20:13). 
This  is  contrary  to  the  military  belief  that  rank  automatically  makes  an 
Air  Force  Officer  an  instant  leader  and  a  superior  manager  (16:32).  In 
public  service  and  government  organizations,  regular  feedback  to 

managers  is  slower  and  less  specific.  This  makes  it  all  the  more 
important  that  we  have  the  information  available  to  make  good 


productivity  choices  (18:40).  The  following  conclusion  came  out  in  a 

study  for  the  Navy  done  by  Litton  System,  Inc.: 

Accurate  productivity  measurements  are  powerful  tools  for  any 
military  or  civilian  manager;  however,  these  tools  are  more 
difficult  to  develop  within  the  military  context  for  there  is 
no  prof it-and-loss  statement;  also,  a  wide  variety  of  policy 
constraints  make  an  overall  prof it-and-loss  type  of 
productivity  measurement  difficult.  Some  type  of  productivty 
measure,  however,  is  essential  to  assist  management.  [26:2-1] 


With  the  current  Department  of  Defense  productivity  programs,  the  H 
obvious  question  that  arises  is,  "How  will  agencies  within  the  Defense  1 
Department  know  if  they  are  or  are  not  improving  productivity?"  The  I 
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answer  is,  of  course,  they  will  not  without  the  initiation  of  some 
measurement  system  (18:3).  The  development  of  meaningful  productivity 
measurement  systems  is  the  key  issue  in  the  Defense  Department 
productivity  programs.  Without  the  measures,  the  agencies  of  DOD  have 
no  hope  of  defining  the  present  level  of  productivity,  nor  can  any 
estimates  of  improvement  or  regression  be  made.  With  no  measurements, 
the  success  or  failure  of  new  management  productivity  improvement 
efforts  cannot  be  assessed  (18:7).  Combine  productivity  measurement 
with  some  old-fashioned  belt-tightening,  and  significant  savings  in 
defense  programs  can  be  acheived  (18:4). 

The  Base  Civil  Engineering  Squadron  never  seems  to  have  sufficient 

resources  to  accomplish  everything  called  for  in  the  mission  statement 

(10:4).  This  limitation  makes  productivty  measurement  within  Civil 
« 

Engineering  a  neccessity.  Lieutenant  Colonel  Norwood  J.  King,  in  his 
article  "How  to  Increase  Work  Force  Productivity,”  stressed  the 
importance  of  being  able  to  understand,  measure,  and  increase 
productivity  in  USAF  Civil  Engineering  (21:8).  The  importance  to  a 
Civil  Engineering  manager  was  summed  up  in  a  thesis  completed  by 
Bauragartel  and  Johnson: 

Base  level  civil  engineering  managers  must  be  able  to  assess  an 
increase  or  decrease  in  productivity  in  order  to  identify  the 
degree  of  attainment  of  this  directorate  goal.  Therefore,  a 
method  of  measuring  the  productivity  of  a  base  Ieve.1  civil 
engineering  organization  Is  definitely  needed.  [3:21] 

Given,  then,  the  importance  of  productivity  measurement  to  managers,  the 

development  of  a  productivity  measurement  model  would  be  of  significant 

value  to  the  Department  of  Defense  and  the  Air  Force  productivity 

programs . 
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Problem  Statement 


In  the  private  sector,  a  productivity  measurement  system  has  been 
developed  based  on  profit  and  economic  standing  in  the  market.  However, 
federal  organizations  do  not  produce  for  profit,  nor  do  they  compete  in 
the  private  sector  markets.  A  new  or  different  method  is  needed  as  it 
is  not  feasible  to  measure  federal  productivity  in  the  same  manner  as 
the  private  sector  (18:3).  If  a  simple  ratio  of  work  output  to  input 
could  be  established,  the  problem  would  be  solved.  Unfortunately,  that 
ratio  is  hard  to  apply  in  service  organizations,  and  the  simplicity 
soon  disappears.  The  mission  of  the  Base  Civil  Engineering  Squadron  is, 
"...to  acquire,  construct,  maintain,  and  operate  real  property 
facilities,  and  provide  related  management,  engineering,  and  other 

support  work  and  services  (10:1).“  In  other  words,  the  mission  is 

# 

service.  Simon,  in  his  book,  Administrative  Behavior:  A  Study  of 
Decision-Making  Processes  In  Administrative.  Organizations,  wrote 
that  for  public  service  organizations,  the  efficiency  is  measured  by  a 
statement  of  objectives  for  that  activity  (33:175).  In  service 

organizations,  objectives  are  stated  social  or  appropriate  substantive 
results  (20:1).  In  government,  this  means  managers  must  attempt  to 

substitute  the  profit  measures  with  intangible  goals  such  as  "national 
defense  (20:1)."  A  study  by  H.G.  Rainey,  comparing  government 
organizations  to  private  industry,  showed  the  following  differences  in 
government  organizations: 

1.  Greater  multiplicity  and  diversity  of  objectives 

2.  Greater  vagueness  and  intangibility  of  objectives 

3.  Greater  tendency  of  goals  to  be  conflicting 

4.  Greater  caution  and  rigidity,  less  innovativeness 
[30:233-244] 


These  differences  make  the  objectives  hard  to  define  and  even  harder  to 


measure . 

The  correct  selection  of  objectives  and  appropriate  criteria  is  a 
critical  step  in  trying  to  determine  productivity.  The  accuracy  and  the 
raeaningf ulness  of  any  productivity  measure  depends  on  the  accuracy  of 
the  measurement  of  the  respective  inputs  and  outputs  and  on  the 
appropriateness  of  the  measurement  units  (26:3-2). 

The  main  problem  is  determining  the  proper  input  and  output 
measures  for  productivity.  Inputs  are  usually  more  easily  determined 
than  outputs  in  service  organizations.  The  measures  of  performance  and 
productivity  must  be  consistent  and  meaningful  if  they  are  to  be  any 
benefit  to  managers.  Productivity  should  not  be  measured  just  for  the 
sake  of  measuring  (16:31).  Only  if  the  productivity  measure  is  used  by 
managers  to  Improve  the  organization  is  the  effort  worthwhile. 

Not  all  people  feel  productivity  should  be  measured.  Major  Donald 
Fowler,  in  a  report  on  white  collar  productivity  measurement,  felt  that 
productivity  in  government  offices  should  not  be  measured.  Rather,  he 
said  productivity  should  be  "assessed”  at  the  macro  level  (16:31). 
While  not  widely  accepted,  Major  Fowler's  assessment  idea  does  not  seem 
so  far-fetched  since  performance  measurement  techniques  in  use  in  AF 
Civil  Engineering  are  almost  entirely  subjective  (10:ili).  This  is 
because,  presently,  there  is  no  acceptable  objective  measurement 
technique.  The  Engineering  Design  Section  of  Civil  Engineering  offers 
some  special  problems.  The  section  is  dominated  by  professionals,  and 
professionals  tend  to  resent  having  their  productivity  measured  (9:21). 
It  is  difficult  to  evaluate  efficiency  and  effectiveness  of  an 


organization  whose  work  deals  with  such  factors  as  design  safety  and 
economy  (9:21). 

It  Is  obvious  that  some  organizations  are  more  effective  than 
others.  The  problem  is  how  to  measure  this  against  different  groups  or 
the  same  group,  taken  at  different  times  (24:1).  Accurate  productivity 
measurements  are  powerful  tools  for  all  managers.  The  military  and 
service  context  of  Base  Level  Air  Force  Civil  Engineering  makes 
productivity  measurement  difficult.  The  problem  is  compounded  by  the 
unique  nature  of  the  engineering  section  within  civil  engineering. 
Presently,  there  is  no  productivity  measurement  method  in  use  within  the 
Air  Force  Civil  Engineering  Design  Section.  Therein  lies  the  problem 
encountered  by  this  thesis,  to  develop  a  managerial  measurement  method 
for  productivity  in  Base  Level  Civil  Engineering  Design  Section. 

Research  Objectives 

The  objectives  of  this  research  are  to:  1)  determine  Air  Force 
Civil  Engineering  Design  Section  outputs  and  inputs  for  productivity 
measures,  2)  determine  an  appropriate  objective  productivity  measurement 
technique,  and  3)  develop  a  productivity  measurement  model  for  a  Base 
Level  USAF  Civil  Engineering  Design  Section.  The  specific  model  and 
actual  measurement  of  productivity  for  every  study  on  productivity 
varies  according  to  the  study  desired,  the  measurement  approach  taken, 
and  the  type  of  organization  being  studied  (3:14-15).  All  studies,  like 
this  one,  have  in  common  an  attempt  to  improve  organizational 
productivity  through  measurement.  There  has  been  considerable  research 
by  various  U.S.  Air  Force  agencies  on  productivity,  including: 


1)  The  Air  Force  Academy  Behavioral  Science  and 
Leadership  Department's  motivation  studies. 

2)  The  Air  Force  Institute  of  Technology. 

3)  The  Air  Force  Military  Personnel  Center's 
development  of  officer  evaluation  reports. 

4)  The  Air  Force  Directorate  of  Personnel  Plan's 
Human  Resources  Development  laboratories. 

5)  The  Leadership  Management  Development  Center's 
(LMDC)  problem  solving  support. 

6)  The  Logistics  Management  Center's  (LMC)  efforts 
to  improve  Air  Force  policies.  [19:1] 


The  area  of  productivity  and  its  relationships  are  constantly 
changing  and  growing.  The  efforts  of  this  research  will  help  future 
researchers  and  managers  better  understand  productivity  and  its 
relationships.  If  a  model  is  developed,  it  will  help  the  design 
responsibility  center/cost  center  fully  accomplish  the  objectives  of  the 
Resources  Management  System  <RMS),  by  enabling  the  manager  to  equate 
resources  consumed  to  output  realized  (15:1). 


Research  Questions 

In  order  to  develop  a  meaningful  measure  of  productivity  for  the 
Base  Civil  Engineering  Design  Section,  the  following  research  questions 
must  be  answered: 

1)  What  are  inputs  and  outputs  for  a  base  level  USAF 
Civil  Engineering  Design  Section? 

2)  What  is  the  appropriate  productivity  measurement 
technique? 

3)  Can  a  model  of  productivity  be  developed  using 
appropriate  inputs  and  outputs? 


Scope  and  Limitations 


This  research  effort  is  focusing  on  the  design  section  of  Air 
Force  Civil  Engineering  in  the  development  of  a  productivity 
measurement  model.  If  the  model  or  its  development  method  can  be 
applied  to  other  responsibility  centers/cost  centers  within  or  outside 
of  civil  engineering,  so  much  the  better.  The  goal  of  this  research 
however,  is  not  to  develop  a  universal  model  but  one  that  is  tailored 
and  specifically  able  to  meet  the  unique  requirements  of  the  design 
section. 

A  measurement  model  is  only  as  good  as  the  measurement  of  its 
inputs  and  outputs  (26:3-2).  Also,  all  measurement  is  subjective  to  a 
certain  degree  (10  :lii) .  It  is  not  the  desire  to  arrive  at  a  perfect 
or  ideal  measurement  model.  If  such  a  model  were  possible  to  develop, 
it  would  be  prohibitively  expensive  to  run.  The  achievement  of  such 
a  model  could  mean  many  managers  would  be  replaced  by  computers 
(26:3-1).  The  desirability  of  this  occurance  is  questionable. 

Assumptions 

We  will  assume  that  the  organizational  goals  or  objectives  for 
the  Civil  Engineering  Design  Section  have  not  changed  significantly 
since  Bauragartel  and  Johnson  identified  them  as  the  following: 

1.  Facility  Life  Cycle  Cost 

A.  Identify  and  program  Military  Construction 

Projects  (MCP)  projects,  and  monitor  approval, 
design  and  construction  phases  to  ensure 
maximum  durability  and  maintainability  of 
accepted  facilities. 

3.  Ensure  in-house  design  complies  with  AFM  88-15 
and  applicable  building  codes. 


2. 
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Facility  Function 

A.  Ensure  new  construction  projects  are 
identified,  programmed  and  designed  in  a 
timely  manner,  and  are  designed  and  located 
in  accordance  with  the  user's  requirements. 

B.  Identify,  program,  and  design  contract 
corrections  to  facilities  which  are 
functionally  inadequate  for  mission 
requirements . 


3.  Facility  Protection 

A.  Ensure  corrective  contract  actions  for 
identified  facility,  fire,  safety,  and 
security  deficiencies  are  programmed, 
designed,  and  completed  in  a  timely  manner. 

B.  Ensure  new  contract  work  complies  with 
regional  requirements  for  structural 
protection  against  weather  and  earthquake 
-related  forces. 


4.  Facility  Occupant/User  Requirements 

A.  Complete  architectural  studies  of  facilities 
to  identify  inadequate  aesthetic  conditions 
and  facility  deficiencies  contributing  to 
occupant  discomfort. 

B.  Ensure  designed  projects  comply  with 
applicable  life  safety  and  public  health  code 
requirements . 

C.  Ensure  identified  facility  life  safety  and 
health  code  deficiencies  requiring  contract 
corrective  actions  are  programmed,  designed, 
and  completed  in  a  timely  manner. 

D.  Identify,  program,  and  specify  custodial 
contracts  required  for  base  facilities  and 
ensure  contractor  compliance  with  the 
contractual  requirements. 

5.  Other  Non-Facility  Requirements 

A.  Provide  professional  architectural  and 

engineering  assistance  to  operations  branch 
and  to  other  organizations  as  required. 
[3:82-83] 


These  objectives  will  be  used  as  the  organizations  goals  in  this 
research  effort. 
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Definitions 


The  definitions  and  terminology  used  in  the  field  of  productiv.  >» 
vary  according  to  the  author.  To  eliminate  any  conflicts  or  confus.  n 
that  may  arise,  the  following  definitions  will  be  used  for  tl 
research  effort.  These  definitions  are  a  compilation  of  th<  e 

currently  in  use  in  productivity  literature  ( 3 ; 35 )  and  are  compatil  :  ' 
with  those  definitions  used  by  Kaneda  and  Wallett  (20),  whose  reseai  h 
helped  form  the  basis  of  this  study. 

INPUT  —  the  quantity  of  resources  used  by  an  organizat '  -n 
during  a  specified  period  of  time.  Inputs  can  lncli 
personnel,  facilities,  energy,  dollars,  raw  materials,  supplit  , 
and  Information. 

OUTPUT  —  the  quantity  of  goods,  products,  and  servic 
produced  or  provided  during  a  specified  period  of  time. 

EFFICIENCY  —  the  ratio  of  output  to  input;  implJ 
minimizing  resource  comsumption  or  maximizing  output  for  giv  n 
resources.  This  terra  does  not  imply  the  appropriateness  of  'he 
output  to  goal  attainment. 

OBJECTIVES  —  the  desired  future  conditions  that  are 
subgoals  of  organizational  goals  which  an  organization  ->r 
section  wants  to  achieve  through  its  activities. 

GOALS  —  the  organizational  goals  that  relate  t  he 

activities  of  an  organization  to  its  environment. 

SURROGATE  MEASURES  —  measures  that  do  not  directly  measur- 
an  aspect  of  efficiency,  effectiveness,  or  productivity,  1" 
research  has  shown  are  indicators  of  actual  performance. 


PRODUCTIVITY  —  efficiency  of  an  organization  in  a  goal 
related  direction.  Given  that  an  organization  is  meeting  its 
mission,  productivity  can  be  measured  as  a  ratio  of  output  to 
input. 


Summary 


Productivity 

in 

the 

United 

States  J 

eclining.  We 

have 

slipped 

from  the  leader 

of 

the 

world 

to  eighth 

among  nations. 

The 

decline 

will  continue  unless  we  do  something  about  it.  The  economy  of  the 
United  States  and  our  standard  of  living  will  drop  without  an  increase 
in  productivity.  Recognizing  this,  DOD  leaders  have  directed  the 
implementation  of  a  DOD  Productivity  Program.  The  key  to  any 
productivity  program  is  the  development  of  a  productivity  measurement 
system.  How  will  we  know  when  the  highest  level  of  performance  has 
been  reached  with  the  least  expenditure  of  resources?  Obviously,  a 
measurement  technique  is  needed. 

Productivity  measures  can  be  a  powerful  tool  for  managers  in  a 
military  organization.  However,  outputs  for  a  service-oriented, 
military  organization  such  as  civil  engineering  are  vague  or  difficult 
to  define.  It  is  obvious  some  organizations  are  more  productive  than 
others.  The  key  is  developing  a  productivity  measurement  model  with 
which  to  evaluate  an  organization's  productivity.  Presently,  there  is 
no  productivity  measurement  model  for  the  design  section  of  Air  Force 
Civil  Engineering. 

Productivity  will  ultimately  lead  to  better  management  in  current 
and  future  operations.  However,  before  this  can  be  realized  an 
appropriate  productivity  measurement  model  must  be  developed. 
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II.  Review  of  the  Literature 


A  review  of  the  literature  concerning  productivity  revealed  a 
plethora  of  views  on  productivity.  Each  article  has  a  slightly 
different  definition  of  productivity.  Rather  than  review  and  comment 
on  the  extensive  amount  of  literature  concerning  productivity  in 
general,  the  authors  chose  to  limit  this  presentation  to  those  works 
thought  to  be  most  applicable  to  this  research  effort.  That  decision 
was  based  on  two  factors:  1)  An  exhaustive  review  of  productivity  in 
general  would  not  contribute  substantially  to  this  report,  and  2)  Such 
exhaustive  reviews  have  been  accomplished  by  others  (we  recommend  the 
technical  report  by  Tuttle  (35)).  Therefore,  this  literature  review 
will  concentrate  on  the  five  articles  the  authors  felt  were  most 
relevant  to  the  task  at  hand. 

POD  Productivity  Program 

The  DOD  established  this  program  in  1975,  with  the  instruction 
that  each  DOD  Component  implement  a  Department/ Agency-wide 
Productivity  Program  (required  by  DOD  Directive  5010.31)  (13).  The 
objective  of  the  program  is  to  obtain  maximum  productivity  growth 
(i.e.  increase  goods  produced  or  services  rendered  relative  to  the 
resources  used)  to  help  offset  personnel  cuts,  reduce  costs,  and  free 
funds  for  other  requirements. 

DOD  first  defined  productivity  as  a  combination  of  efficiency  and 
effectiveness,  saying  that  organizations  must  be  efficient  (accomplish 
the  right  things  at  the  lowest  expense)  and  effective  (accomplish  the 


at  the 


right  things  at  the  right  times).  Thus,  "The  efficiency  with  which 
organizations  utilize  all  types  of  fund  resources  to  accomplish  their 
mission  represents  total  resource  productivity  (13:1)."  Later  in  the 
same  instruction  documents,  DOD  states  that  a  productivity  index  is 
"the  percentage  ratio  of  goods  produced  or  services  rendered  (outputs) 
to  resources  expended  (inputs)  during  a  current  period  in  relation  to 
a  base  period  (13:11)."  This  second  definition  of  productivity  is  the 
same  as  our  definition  of  efficiency;  thus,  the  official  DOD 
definition  of  productivity  is  unclear. 

The  instruction  goes  on  to  list  the  minimum  department  functions 
that  must  provide  productivity  indicator  data.  The  instruction  also 
suggests  possible  indicators  for  each  department  function.  The  Air 
Force  Civil  Engineering  Design  Section  is  not  specified  in  the 
instruction  (13:Encl  3:4-8). 


Hanley  and  Smith 

A  1976  AFIT  thesis  by  the  research  team  of  Hanley  and  Smith 
analyzed  the  effect  of  labor  manhour  requirement  estimates  on  the 
measurement  of  Air  Force  Civil  Engineering  (AFCE)  productivity  (17:7). 
Their  findings  indicated  that  significant  variation  existed  in  the 
standard  manhour  requirements  estimates  being  computed  by  Air  Force 
planners  for  any  given  project  (17:58).  Those  estimates  were  the 
standards  against  which  actual  labor  manhours  were  compared  to  compute 
a  productivity  index.  Since  the  existence  of  variation  would  result 
in  different  standard  estimates  for  the  same  project  at  different 
bases,  Hanley  and  Smith  concluded  that,  "Comparisons  of  standard 


estimates  with  actual  labor  manhour  expenditures  result  in  unreliable 
productivity  ratios  (17:73)." 

Baumgartel  and  Johnson 

In  1979,  the  AFIT  research  team  of  Baumgartel  and  Johnson 
attempted  to  develop  productivity  measures  for  a  USAF  base  level  civil 
engineering  organization  (3).  They  defined  productivity  as  "the 
measure  of  effective  and  efficient  use  of  resources  to  attain  results 
which  are  directed  towards  the  strategic  level  organizational  goals, 
through  the  branch  level  objectives  (3:24)."  In  their  model,  they 
proposed  to  measure  productivity  by  taking  the  average  value  of  the 
performance  indicators  for  each  branch  objective,  divided  by  the  total 
resources  used  to  attain  the  level  of  output  (3:24).  The  resulting 
ratio  of  performance  achieved  to  resources  consumed  represented  that 
branch's  effort  to  support  a  specific  objective  during  that  specified 
time  period;  a  series  of  measurements  taken  over  time  would  be  an 
indicator  of  any  change  in  the  branch's  productivity  (3:71-72). 

It  should  be  noted  that  their  measurements  would  require  a  length 
of  time  before  any  Interpretation  of  the  ratio  would  be  possible. 
Additionally,  each  organization  would  be  measured  against  itself;  no 
mention  is  made  of  the  ability  to  compare  one  organization  to  another 
based  on  the  ratios. 

Baumgartel  and  Johnson  concluded  that  while  the  input  data  was 
available  in  great  quantity  and  detail,  the  output  data  needed  for 
their  index  calculations  was  limited.  Using  their  defined  objectives, 
they  found  it  difficult  to  classify  the  output  information.  They 


could  net  measure  readiness  or  response  output,  and  they  questioned 
the  inclusion  of  any  training  or  exercise  evaluation  results  (3:108). 

Tuttle 

In  Thomas  Tuttle's  extensive  review  of  productivity  implications 
for  the  Air  Force,  he  offers  five  different  definitions  of 
productivity,  three  of  which  bear  mention.  The  economist  defines 
productivity  as  the  ratio  of  output  to  input  when  both  output  and 
input  are  measured  in  real  (physical  volume)  terms  (35:7).  The 
engineer,  on  the  other  hand,  defines  productivity  from  the  idea  of 
mechanical  efficiency  (35:8): 


Productivity  (efficiency)- 
or 


Useful  Hork 
Energy 


stated  another  way:  Productivity-  Useful  Output 


Input 


In  comparing  the  engineer's  definition  of  productivity  to  the 
economist's,  three  important  distinctions  should  be  made.  First,  the 
upper  limit  of  efficiency  for  the  engineering  definition  is  unity 
(one),  while  In  the  financial  definition,  the  ratio  can  and  should 
exceed  1.  Second,  the  engineering  definition  distinguishes  between 
"total"  output  and  "useful"  output  (i.e.  goal  directed,  quality 
output)  (35:9).  Finally,  the  purpose  of  the  engineering  definiton  is 
to  measure  individual  or  small  unit  operations,  while  the  economist's 
approach  is  to  develop  statistics  for  comparing  total  industries 


The  manager  appears  to  have  a  broader  interpretation  of 
productivity.  Based  on  a  nationwide  survey  of  Chief  Executive 
Officers  (CEO)  and  Industrial  Relations  Officers  on  their  definitions 
of  productivity,  "Virtu  illy  9  out  of  10  managers  would  include 
quality,  effectiveness  and  efficiency  in  their  definition;  7  out  of  10 
would  add  the  idea  of  work  stoppages,  waste,  shrinkage,  sabotage, 
absenteeism  and  turnover;  and  6  cut  of  10  managers  would  include 
measures  of  customer  or  client  satisfaction  (35:11).”  From  that,  one 
can  conclude  that  there  is  a  wide  difference  of  opinion  as  to  the 
manager's  definition  of  productivity. 

Tuttle  concludes  that  an  Air  Force  organization's  definition  of 

productivity  should  incorporate  the  concepts  of  both  efficiency  and 

effectiveness.  Also,  auy  productivity  measurement  scheme  for  an 
* 

organization  should  include  multiple  measures  of  both  efficiency  and 
effectiveness  (35:76).  Finally,  Tuttle  identifies  seven  desireable 
characteristics  of  a  set  of  productivity  measures: 

1.  Completeness  -  the  set  of  measures  adequately  covers 
the  significant  facets  of  the  organization's  mission. 

2.  Comparability  -  the  measures  should  be  applicable 
over  time  to  permit  longitudinal  measurements  of  productivity 
within  the  organization. 

3.  Input  coverage  -  outputs  used  In  measurement 
represent  all  the  relevant  Inputs. 

4.  Compatibility  with  existing  data  sources  -  attempt  to 
use  data  already  available. 

5.  Cost  effectiveness  -  the  benefits  derived  should 
exceed  the  costs  incurred  in  making  the  measurements. 

6.  Consistent  across  organizations  -  the  most  useful 
measures  would  be  relatively  invariate  across  organizations 
performing  the  same  function. 


7.  Acceptable  to  organization  members  -  the  managers  and 
workers  being  measured  accept  and  support  the  measures. 
[35:77-78] 


Kaneda  and  Wallett 

In  1980,  the  AFIT  research  team  of  Kaneda  and  Wallett  set  out  to 
develop  productivity  measures  for  the  design  section  of  a  base  civil 
engineering  organization.  Using  a  questionnaire  survey  instrument, 
they  collected  data  on  proposed  productivity  measures  from  base  civil 
engineers,  chiefs  of  design,  and  chiefs  of  Industrial  engineering 
throughout  the  United  States.  Statistical  aualyses  then  yielded  six 
measurements  acceptable  to  the  majority  of  survey  respondents.  The 
six  productivity  measures  are: 

1.  Total  estimated  dollar  amount  of  contract  projects 
and  in-house  work  orders  designed  divided  by  total  design 
manhours . 

2.  Total  number  of  projects  designed  (complete  and  ready 
for  acquisition  action)  divided  by  total  design  manhours. 

3.  Total  number  of  facility  inspections  and  utility 
systems  surveys  completed  divided  by  total  manhours  to 
complete  surveys  and  inspections. 

4.  Total  estimated  dollar  amount  of  architect-engineer 
(A-E)  design  acquisition  packages  prepared  divided  by  total 
manhours  to  prepare. 

5.  Total  estimated  dollar  amount  of  contract  projects 
and  in-house  work  orders  designed  divided  by  total  design 
labor  cost. 

6.  Total  number  of  projects  designed  (complete  and  ready 
for  acquisition  action)  divided  by  total  design  labor  cost. 
[20:76] 


Two  additional  measures  were  identi  by  two  prominent  partial 
samples  (the  Base  Civil  Engineers  and  t  "  agers  with  over  10  years 


experience  in  a  base  civil  engineering  organization),  and  may  be 
considered  useful  to  the  managers  of  the  design  section: 

1.  Total  contract  funds  obligated  (i.e.  Military 
Construction  Program  and  Operations  and  Maintenance)  divided 
by  total  design  manhours  associated  with  the  contract  funds 
obligated. 

2.  Number  of  work  orders  reviewed  and/or  evaluated 
divided  by  total  manhours  required  for  review  and  or 
evaluation.  [20:77] 

Kaneda  and  Wallett  concluded  that  those  productivity  measurements 
should  be  useful  as  a  starting  point  to  measure  productivity  trends. 
They  also  stated,  though,  that  every  measurement  may  not  be  applicable 
to  every  design  organization,  and  advised  against  comparing  dissimilar 
design  sections  (i.e.  design  sections  with  dissimilar  projects,  level 
of  eugineer  experience,  etc.  (20:79-80).  < 


Summary 

The  preceding  literature  review  has  been  a  chronological  review 
of  productivity  in  general,  and  a  narrowing  of  the  scope  of 
productivity  measurements  from  DOD  to  Air  Force  to  base  level  Civil 
Engineering  to  the  Civil  Engineering  Design  Section.  Hanley  and  Smith 
showed  that  productivity  should  not  be  measured  as  a  ratio  of  actual 
manhours  to  standard  manhours  required,  as  the  estimates  for  standard 
manhours  required  were  too  varied.  Baumgartel  and  Johnson  then 
proposed  a  model  for  measuring  productivity  of  a  total  base  level 
Civil  Engineering  organization  by  using  performance  indicators  for 
branch  objectives,  divided  by  total  resources  used  to  attain  those 
objectives.  Baumgartel  and  Johnson  could  not  classify  their  outputs 
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by  inputs  consumed,  though.  Kaneda  and  Wallett  then  looked  at  a 


portion  of 
identified 


a  Civil  Engineering  organization — the  design  section — and 
at  least  six  productivity  measures  that  would  be  useful  to 


managers  of  design  sections  in  determining  the  section's  productivity. 


III.  Methodology 


Given  that  productivity  measurement  is  important  and  necessary, 
we  now  will  attempt  to  provide  a  means  for  measuring  productivity  in  a 
Civil  Engineering  Design  Section.  The  desired  end  result  will  be  some 
form  of  a  productivity  model  that  will  permit  the  design  chief  to 
analyze  his  section  and  then  make  managerial  decisions  that  will 
improve  the  performance  of  his  section  based  on  that  analysis. 

Four  steps  must  be  taken  to  obtain  the  desired  result  of  a 
productivity  measurement  for  a  design  section.  First,  we  must 
determine  what  the  actual  inputs  and  outputs  of  a  design  section  are. 
That  is,  we  must  know  all  of  the  resources  that  a  design  section 
receives,  and  we  must  know  what  products,  services,  and  activities 
result  from  the  consumption  of  those  resources.  Second,  we  must  have 
some  method  for  measuring  those  inputs  and  outputs.  The  inputs  and 
outputs  themselves  can  be  measured,  or  they  can  be  measured  indirectly 
by  using  a  surrogate.  Third,  the  input  and  output  measures  must  be 
gathered.  Finally,  some  form  of  mathematical  analysis  of  the  data 
must  be  performed  to  determine  the  unit's  productivity  rating  or 
measurement,  either  with  respect  to  other  design  units  or  with  respect 
to  the  unit  itself  over  a  period  of  time. 

Overview 

A  review  of  the  various  resources  (inputs)  and  products  and 
services  (outputs)  of  a  design  section  yielded  a  list  of  over  thirty 
factors  to  be  considered  in  the  measurement  of  a  design  section's 
productivity.  The  list  was  then  analyzed  and  subsequently  reduced  to 
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a  total  of  twenty-six  inputs  and  outputs  representing  the  work  of  a 
design  section.  Simultaneously,  various  analytical  techniques  were 
examined  for  use  in  analysis  of  the  input  and  output  data.  One 
technique,  Constrained  Facet  Analysis  (CFA),  appeared  to  be  exactly 
the  type  of  model  needed  to  analyze  a  complex  organization  with 
multiple  inputs  and  outputs.  Accordingly,  it  was  selected  as  the 
analysis  technique. 

Due  to  the  selection  of  the  CFA  model  and  its  short  history,  the 
authors  felt  that  a  demonstration  of  the  model  was  appropriate.  Data 
was  generated  for  fifty-two  organizations.  The  data  was  constructed 
so  that  four  organizations  were  definitely  efficient  with  respect  to 
the  othet  organizations,  and  four  units  were  definitely  the  least 
efficient.  A  successful  demonstration  of  the  model  would  occur  if  the 
CFA  technique  verified  the  four  efficient  organizations  and  the  four 
most  inefficient  organizations. 

Analysis  Techniques 

First  of  all,  it  must  be  recognized  that  the  problem  being 
confronted  is  one  of  multiple  inputs  and  multiple  outputs.  That  is,  a 
design  section  does  not  simply  take  manhours  and  turn  them  directly 
into  design  drawings.  Other  inputs  (eg.,  number  of  personnel, 
supplies)  and  outputs  (eg.,  site  inspections,  design  reviews,  etc)  are 
involved.  Therefore,  any  analysis  technique  must  be  capable  of 
handling  both  multiple  inputs  and  multiple  outputs  simultaneously, 
permitting  interdependence  of  factors  to  be  Included  as  part  of  the 
examina  tion. 
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The  next  step  is  to  research  those  techniques  currently  in  use  or 
thought  to  be  applicable  to  the  problem  at  hand.  Three  different 
techniques  were  found  to  be  available.  The  three  analysis  techniques 
chosen  were  those  felt  by  the  authors  to  be  the  most  appropriate  for 
this  problem,  though  certainly  not  exclusive  of  all  others.  The 
techniques  examined  were:  1)  ratio  analysis;  2)  regressioi  analysis; 
and  3)  Data  Envelopment  Analysis  (DEA). 

Upon  examination,  the  Constrained  Facet  Analysis  (CFA) 
technique — an  adaptation  of  DEA — was  selected  as  the  appropriate 
method.  The  three  techniques  will  be  examined  in  the  following  section 
to  present  the  reasons  why  CFA  was  chosen  over  the  other  methods.  Each 

method  will  be  explained  along  with  its  relative  strengths  and 

weaknesses  as  revealed  by  the  research. 

Ratio  Analysis.  Ratio  analysis  is  a  one-to-one  comparison  of 
outputs  to  inputs  used  frequently  today  as  measures  of  productivity  or 

efficiency.  Ratio  analysis  is  popular  because  it  is  easy  to  use  and 

familiar  to  managers  (18).  Managers  are  comfortable  with  a  measurement 
that  conveys  some  tangibility  (eg.,  number  of  widgets  produced  per 
hour).  It  can  be  used  without  knowing  the  production  process  or  when 
the  production  process  J.s  difficult  to  model  (5:23). 

The  problem  with  ratio  analysis  lies  in  its  inability  to  consider 
the  full  range  of  inputs  3nd  outputs  and  their  interactions 
simultaneously  (5:23).  Ratios  typically  consider  one  input  and  one 
output  at  a  time.  Thus,  any  one  ratio  is  only  a  partial  measurement  at 
best  of  an  organization's  activity  (18).  A  number  of  ratios  are  needed 
to  completely  cover  the  range  of  the  organizational  activity,  and  yet 
they  still  do  not  portray  any  of  the  interactions  between  outputs 


competing  for  the  same  resources  or  inputs  dependent  on  one  another. 

Comparisons  between  organizations  are  difficult  when  using  the 
ratio  method,  particularly  when  one  organization  ranks  higher  in  some 
measurements  while  the  other  organization  ranks  higher  in  other 
measurements.  The  ratios  must  then  be  given  some  order  of  importance 
to  permit  comparison  between  the  organizations.  As  the  number  of 


ratios  grows 

(ie., 

the 

number 

of  inputs 

and 

outputs  to  be  compared 

increases) , 

the 

task 

of 

weighting 

and 

assimilation  grows 

multiplicatively  (5:23). 

Regression  Analysis.  Regression  analysis  attempts  to  find  a 
relationship  between  some  response  variable  (productivity)  and  some  set 
of  secondary  variables  (productivity  indicators).  The  objective  is  to 
find  a  linear  equation  for  the  relationship  so  that  the  secondary 
variables  can  be  used  to  better  predict  or  estimate  the  response 
variable  (productivity)  (25:523). 

Least  squares  regression  is  defined  as  that  curve  (line)  which 
minimizes  the  sura  of  squares  of  the  lengths  of  vertical  line  segments 
drawn  from  the  observed  data  points  on  the  scatter  diagram  to  the 
fitted  curve  (22:45).  It  is  important  to  note  that  a  least  squares 

regression  of  a  single  output,  multiple  input  case  having  both  positive 

and  negative  error  terms  will  produce  a  curve  (line)  of  average 
relationship  (5:18).  The  curves  are  not  frontier  or  extremal 

relationships,  as  actual  outputs  can  lie  above  or  below  the  curve. 

In  the  multiple  output  case,  regression  analysis  must  be  performed 
on  each  output  separately.  As  in  the  ratio  analysis,  this  prevents 
consideration  of  the  impact  of  interdependencies  or  competition  for 

resources  among  the  variables.  In  their  paper  "Managerial  Efficiency 


Measurement,  Theory  and  Applications,"  Bessent,  Bessent,  and  Cla 
explain  the  impact  of  this  deficiency  as  follows: 

Each  regression  equation  might  be  able  to  predict 
adequately  an  expected  level  of  a  single  output  for  an 
organization,  assuming  this  organization  could  experience  an / 
of  the  random  fluctuations  or  inefficiencies  of  the  industry 
(all  firms)  and  recognizing  that  the  influence  of  other 
outputs  are  implicitly  taken  into  account  by  the  deviations 
from  the  regression  line  (residuals).  But  these  equations 
cannot  predict  the  expected  output  of  an  organization  whose 
variations  and/or  inefficiencies  are  significantly  affected 
by  the  given  technology  and  policies  of  the  firm  which  are 
not  random.  Magnitudes  of  actual  outputs  of  an  organization 
are  influenced  by  both  local  and  corporate  policy  which  may 
prevent  the  true  expected  output  values  of  the  orgnization 
from  conforming  to  the  corresponding  regression  estimates. 
[5:19] 

If  it  is  accepted  that  a  linear  regression  model  is  appropriat 
then  the  regression  estimates  must  reflect  an  efficient  input/outp 
relationship  (5:20).  Therefore,  the  rate  of  technical  substitution  1 
assumed  to  be  constant  and  the  rate  at  which  an  input  is  converted  ini 
an  output  is  the  same  for  all  of  the  organizations  in  the  model  (5:20 
That  means  that  an  organization  cannot  decrease  one  input  witho 
increasing  another  input,  or  total  output  must  decrease.  Also,  i 
organization  cannot  increase  output  without  some  increase  in  Inputs. 

i 

While  those  statements  are  true  of  the  efficient  organization 
inefficient  organizations  should  be  able  to  increase  output  withon 
increasing  inputs  or  decrease  inputs  while  holding  output  constant 
increase  efficiency. 

Data  Envelopment  Analysis  (PEA).  Data  Envelopment  Analysis  is 
a  relatively  new  method  of  evaluating  productivity  in  not-f or-prof 
organizations.  After  researching  the  other  available  analysis  method 
we  concluded  that  DEA  offered  the  best  possibility  of  being  able 
measure  productivity  in  a  Civil  Engineering  Design  Section.  A  memb 
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of  the  AFIT  faculty,  Lt  Col  Charles  T4  Clark,  has  been  very  active  in 
pioneer  work  with  the  DEA  model  as  part  of  his  Doctoral  effort,  and  has 
been  of  great  assistance  to  this  research  team  in  explaining  the 
procedures  and  results  of  DEA.  His  assistance  was  particularly 
valuable  because  of  the  extremely  small  body  of  literature  available  on 
DEA.  Therefore,  the  following  discussion  of  DEA  and  its  offspring, 
Constrained  Facet  Analysis  (CFA),  is  drawn  from  class  lectures  from  Lt 
Col  Clark  and  two  papers  which  he  contributed  to. 

Data  Envelopment  Analysis  is  a  fractional  programming  model  for 
measuring  the  efficiency  of  similar  management  units  (5:1).  The  model 
is  capable  of  taking  into  account  both  multiple  inputs  and  multiple 
outputs;  it  then  calculates  an  efficiency  rating  for  each  unit  relative 
to  other  similar  units  that  have  produced  greater  outputs  with  their 
Inputs. 

The  approach  to  DEA  was  conceptualized  by  M.  J.  Farrell  in  1957, 
in  a  paper  entitled,  "The  Measurement  of  Productive  Efficiency  (8:3)." 
His  work  provided  a  major  breakthrough  in  specifying  a  frontier  of 
relative  efficiency  while  not  assuming  any  particular  form  for  the 
industry  production  function  (5:4).  Subsequently,  work  done  in  1978  by 
Charnes,  Cooper,  and  Rhodes  solved  some  mathematical  computational 
problems  in  Farrell's  work,  and  it  was  their  operational  linear 
programming  model  which  they  named  "Data  Envelopment  Analysis  (8:3)." 
Computer  software  for  solution  of  the  model  was  provided  by  the  team  of 
fli,  Bessent,  Bessent,  and  Kennington,  and  field  applications  began  in 
1980  (3:3). 
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Figure  1.  Farrell's  Efficiency  Isoquant 

Figure  1  represents  Farrell's  concept  of  an  efficiency 

frontier — an  isoquant  along  which  various  combinations  of  inputs 
Xj  and  *2  are  all  equally  efficient  in  producing  one  unit  of 
output.  Again,  this  isoquant  Is  established  with  observed  units 
operating  at  maximum  relative  efficiency.  Any  inefficient  unit  must 
use  more  of  one  or  both  of  the  inputs.  Therefore,  no  observations  can 
fall  between  the  isoquant  and  the  origin.  Inefficient  units  will 
appear  somewhere  above  the  isoquant,  as  illustrated  by  point  p. 
Efficient  units  appear  on  the  isoquant,  as  represented  by  point  q. 
The  distances  from  the  origin  to  point  p  and  q,  respectively,  can  be 
represented  by  the  line  segments  5p  and  Sq.  Farrell  defined  the 
technical  efficiency  of  unit  p  as  the  ratio  oq/5p  (5:4).  That  is, 
unit  p  must  decrease  its  inputs  x^  and  X2  by  the  ratio  oq/ffp 
to  be  equally  efficient  as  unit  q. 
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One  of  the  most  important  points  to  be  made  concerning  DEA  is  the 
idea  that  each  management  unit  is  evaluated  against  all  other  similar 
units  ("similar”  meaning  having  roughly  equal  mixes  of  inputs  and 
outputs).  Those  determined  to  be  most  efficient  are  assigned  a  rating 
of  1.0  and  are  placed  on  a  frontier  which  the  inefficient  units  are 
compared  to.  Thus  there  is  no  artificial  standard  being  used  as  a 
yardstick;  management  units  are  measured  against  the  other  similar 
management  units  (called  a  ’neighborhood”)  in  the  group  with  the  most 
efficient  units  becoming  the  basis  for  comparison.  Observed  standards 
are  used  as  opposed  to  the  usual  theoretical  standards.  Farrell,  as 
quoted  in  Bessent,  Eessent,  and  Clark,  noted  that,  "...it  is  far 
better  to  compare  performances  with  the  best  actually  achieved  than 
with  some  unattaiuable  ideal  (5:4).” 

Some  requirements  must  be  met  by  the  organizations  being  measured 
before  DEA  can  be  used.  First  of  all,  organizations  must  be  using 
varying  amounts  of  the  same  resources  (5:14).  The  authors  believe  CE 
design  sections  meet  that  criterion.  All  design  sections  are  composed 
of  some  mix  of  military  and  civilian  personnel  and  receive  various 
amounts  of  funding  from  a  central  authority.  Every  design  section  is 
responsible  for  the  same  tasks,  although  their  actual  output  varies 
from  base  to  base.  Thus,  the  design  sections  are  sufficiently  similar 
to  satisfy  the  first  requirement.  Second,  it  is  important  to  select 
as  many  inputs  and  outputs  as  possible  depending  on  how  many 
organizations  are  being  measured  (5:14).  Input  and  output  selection 
is  discussed  later  in  this  chapter;  we  will  use  at  least  twice  as  many 
organizations  as  the  total  of  inputs  and  outputs  chosen. 
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It  is  also  important  to  observe  the  following  rules,  as  outlined 
in  Bessent,  Bessent,  and  Clark  (5): 

1.  All  measures  should  be  total  quantities  or  ratios  which 
have  a  common  denominator. 

2.  All  units  must  use  some  amount  of  each  input  to  produce 
some  amount  of  each  output. 

3.  Input  measures  should  be  selected  that  have  a  positive 
relation  to  output— that  is,  an  increase  in  the  input  should 
cause  an  increase  in  output. 

4.  Regardless  of  how  a  resource  is  acquired,  it  should  be 
included  In  the  analysis  if  it  has  some  affect  on  the 
production  of  outputs. 

While  the  Data  Envelopment  Analysis  technique  is  effective  in 
identifying  efficient  and  inefficient  organizations,  it  is  limited  in 
its  ability  to  provide  planning  information  (8:2-3).  Work  by  Clark  at 
the  University  of  Texas  resulted  in  a  new  method  of  computing 

I 

efficiency  called  Constrained  Facet  Analysis  (CFA),  which  was  then 
sucessfully  tested  in  field  work  by  Bessent  and  Bessent  (8:3).  The 
extensions  of  efficiency  analysis  developed  in  CFA  allow  the  managers 
to  Identify  those  organizations  which  had  similar  inputs  but  higher 
outputs  (ie.,  those  that  were  more  efficient)  (8:4).  They  are  then 
rank  ordered  in  terms  of  similarity  and  available  to  the  manager  of 
the  less  efficient  organization  for  review.  That  manager  can  then 
talk  to  or  visit  the  more  efficient  organizations  to  determine  what  it 
is  that  makes  them  more  efficient. 

Constrained  Facet  Analysis  is  a  normative  type  model,  meaning 
that  it  performs  optimization  calculations.  As  such,  it  is  an 
abstract  mathematical  model  that  is  very  difficult  t.o  understand  and 
validate  (4).  CFA  has  been  used  successfully  in  three  areas:  1)  the 
Texas  Public  School  system;  2)  aircraft  maintenance,  and  3)  a  mental 
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hospital  (4).  Its  limited  use  to  this  point  is  explained  by  its  short 
exis  tence . 

The  CFA  model,  as  presented  in  the  paper  by  Bessent,  Bessent, 
Clark,  and  Elam,  is  shown  in  Appendix  A  (8:16-19). 

Summary  of  Analysis  Techniques.  The  Constrained  Facet 
Analysis  (CFA)  model  has  been  chosen  oyer  ratio  analysis  and 
regression  analysis.  CFA  is  an  offspring  of  Data  Envelopment  Analysis 
(DEA)  that  improves  the  management  Information  provided  by  DEa. 

The  CFA  model  exhibits  the  following  desirable  characteristics: 

1.  The  model  simultaneously  considers  multiple 
inputs  and  multiple  outputs. 

2.  Results  in  aggregate  measures  of  efficiency 
for  each  organization. 

3.  Units  are  evaluated  as  efficient  or 
inefficient  relative  to  a  neighborhood 
frontier  region  of  actual  achievement. 

4.  The  maximum  achievable  efficiency  value  is 
one  (1.0). 

5.  The  inputs  and  outputs  do  not  require  common 
scales  or  units  of  measurement 

6.  Indicates  input  overages 

7.  Indicates  output  shortages 

3.  The  information  provided  is  a  major 

improvement  over  those  models  currently 
used  (ie.,  ratio  and  regression)  in  the 
measurement  of  public  service  productivity. 

Input  and  Output  Determination 

The  approach  in  determining  inputs  and  outputs  for  the 
Constrained  Facet  Analysis  model  is  outlined  in  a  paper  by  Bessent, 
Bessent,  and  Clark  entitled  "Specification  of  Inputs  and  Outputs  (6)." 
As  a  prerequisite  for  the  CFA  model,  both  the  necessary  data  and  the 


model  itself  should  be  able  to  interface  with  the  existing  management 
information  system.  The  data  rhould  be  available  from  the  information 
currently  compiled  or  recorded  in  the  base  level  Civil  Engineering 
organization  to  avoid  any  additional  administrative  workload. 

The  current  management  information  system  in  use  in  the  Civil 
Engineering  community  is  the  Base  Engineer  Automated  Management  System 
(BEAMS).  This  system  has  been  described  as  probably  the  most 
comprehensive  performance  reporting  system  in  use  in  the  Air  Force 
today  (20:33).  BEAMS  has  a  huge  historical  data  base.  One  of  the 
purposes  of  BEAMS,  as  outlined  in  Air  Force  Manual  171-20'.  is  to 
allow  more  efficient  and  effective  management  of  resources  by  the  base 
Civil  Engineering  organizations  (31:1). 

BEAMS  has  an  outline  subsystem  Interface  that  causus  the  data 

A 

from  a  single  update  transaction  to  be  fed  into  other  files  that 
require  the  same  information  (31:2).  Two  of  these  subsystems  that  are 
important  to  the  base  level  Civil  Engineering  Design  Section  are  the 
Labor  and  Prime  BEEF  Subsystem  and  the  Civil  Engineering  Contract 
Reporting  System  (CECORS).  The  Labor  and  Prime  BEEF  Subsystem 
provides  for  the  accounting  of  authorized  and  assigned  personnel,  the 
reporting  of  labor  expended  against  specific  work  orders,  and  the 
accounting  for  time  other  than  direct  labor  expended  against  specific 
work  orders.  The  Civil  Engineering  Contract  Reporting  System 
maintains  the  scatus  of  all  service  contracts  and  all  active  and 
programmed  projects  regardless  of  fund  source.  Additionally,  CECORS 
produces  a  monthly  report  for  submission  to  the  responsible  MAJCOM, 
and  includes  two  optional  engineering  design  backlog  management 
capabilities  (31:3-4).  The  importance  of  these  subsystems  will  be 
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seen  when  Che  inputs  and  outputs  are  developed.  Additional  data  that 
is  not  maintained  on  these  sybsystems  can  be  obtained  using  an  Air 
Force  On  Line  Data  System  (AFuLDS)  retrieval  that  allows  the  user  to 
specify  exactly  what  data  he  wants  (31:8). 

One  criticism  of  BEAMS  has  been  that  the  data  is  not  current. 
Some  have  questioned  the  integrity  of  the  data.  The  time  lapse  in 
data  permits  examination  of  the  data  prior  to  entry  or  manipulation  to 
make  the  situation  reflect  a  more  favorable  situation  (31:9).  This 
would  not  be  a  problem  on  a  productivity  measurement  model  since 
productivity  is  rarely  fig  red  on  a  daily  basis. 

The  future  of  BEAMS  is  uncertain  with  the  new  proposed  Work 
Information  Management  System  (WIMS).  However,  the  proposal  is  for 
WIMS  to  be  implemented  in  addition  to  BEAMS.  One  thing  that  is 
certain  in  BEAMS'S  future  is  that  it  will  be  aound  for  awhile  with  or 
without  WIMS.  This  is  important  in  the  implementation  of  the 
productivity  measurement  model  since  the  data  must  be  available  on  a 
management  information  system  for  the  model  to  be  useful.  It  would  be 
fruitless  to  develop  a  model  that  would  be  rendered  obsolete  by  the 
abolition  or  replacement  of  the  management  information  system  on  which 
the  data  is  maintained.  This  does  not  appear  to  be  the  case  with  the 
Base  Engineer  Automated  Management  System.  It  will  continue  to  be  in 
existence  in  the  future  with  or  without  a  base  level  Civil  Engineering 
Design  Section  productivity  measurement. 

The  first  step  in  the  determination  of  inputs  and  outputs  (the 
process  is  outlined  in  Figure  2)  is  to  determine  the  decision  making 
unit  (DMU).  The  decision  making  unit  in  this  research  project  is  the 
base  level  Civil  Engineering  Design  Section. 
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Figure  2.  Input  and  Output  Determination  Diagram  [6:?.] 
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Once  the  decision  making  unit  has  been  selected,  the  next  step  is 
to  develop  a  formal  statement  of  the  DMU  goals  and  services.  These 
goals  and  services  will  be  those  organizational  objectives  determined 
by  Baumgartel  and  Johnson  (3)  and  stated  in  Chapter  I. 

The  next  step  is  to  review  all  pertinent  studies  and  reports. 
This  review  was  done  as  part  of  the  literature  review  conducted  in 
Chapter  II  of  this  research  effort.  The  most  applicable  was  a 
research  effort  conducted  by  Kaneda  and  Wallett  (20)  in  a  master's 
thesis  at  the  Air  Force  Institute  of  Technology,  School  of  System  and 
Logistics.  As  part  of  the  research  they  developed  a  suggested 
productivity  measurement  system  for  the  base  level  Design  Section. 
This  leads  to  step  three  which  is  determining  available  measures  or 
current  systems  (6:2).  The  system  developed  by  Kaneda  and  Wallatt  was 
never  adopted  by  the  Air  Force  and  in  not  in  use  today. 

•  The  next  step  is  to  describe  the  managerial  context.  In 
developing  this  managerial  context,  the  organizational  goals, 
operational  requirements,  and  managerial  system  must  be  evaluated 
(6:1).  The  views  of  key  managers,  analysts,  and  technicians  should  be 
sought  while  determining  the  managerial  context  (6:1).  This  was  done 
in  several  ways  by  Kaneda  and  Wallett.  The  first  way  was  using  a 
panel  of  fellow  Facilities  Management  students  at  the  Air  Force 
Institute  of  Technology  (AFIT)  School  of  System  and  Logistics.  This 
panel  consisted  of  five  members  with  5  to  12  years  experience  in  the 
Air  Force  Civil  Engineering  career  field  (20:84).  Each  member  of  the 
panel  had  also  been  associated  with  or  managed  a  base  level  Civil 
Engineering  Design  Section  (20:84).  The  second  way  Kaneda  and  Wallett 
collected  the  views  of  experts  was  using  a  panel  made  up  of  faculty 
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members  from  the  Air  Force  Institute  of  Technology  School  of  Civil 
Engineering  (20:85).  This  panel  of  experts  provided  a  forum  for 
criticism  and  recommendations  for  improvement. 

This  method  was  selected  for  use  in  this  research  effort  also. 
Two  panels  were  selected  and  used  as  a  sounding  board  to  provide 
criticism  and  recommendations.  The  first  panel  consisted  of  AFIT 

Engineering  Management  students  with  at  least  6  years  civil 
engineering  experience  and  some  design  experience.  The  second  panel 
was  comprised  of  AFIT  School  of  Civil  Engineering  faculty.  The 
members  of  the  panels  are  shown  in  Appendix  B. 

The  next  step  in  the  determination  of  inputs  and  outputs  wns  to 
generate  a  comprehensive  set  of  input  and  output  measures.  First  we 
will  discuss  identifying  relevant  input  measures.  Kaneda  and  Wallett 
used  only  direct  labor  cost  and  manhours  (20:54).  This  research 
effort  analyzed  n  number  of  possible  input  measures.  In  generating  a 
comprehensive  list  of  inputs,  a  total  of  six  inputs  were  considered. 
This  list  was  submitted  to  the  two  panels  for  screening  for  relevancy 
and  completeness.  From  this  list,  all  six  inputs  were  selected  for 
inclusion  in  the  model.  These  inputs  are  as  follows: 

1.  Labor  manhours 

2.  Labor  costs 

3.  Years  experience 

4.  Personnel  skill  level  aggregate 

5.  Number  of  professional  educational  courses 
completed 

6.  One  over  the  number  of  additional  duties  performed 
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The  amount  of  equipment  available,  facilities  available,  and  overhead 
were  not  included  in  the  analysis.  These  were  analyzed  by  Kaneda  and 
Wallett,  and  they  concluded  that  these  did  not  vary  significantly  in 
various  design  sections.  The  reason  given  for  this  small  variance  is 
that  all  the  design  sections  are  controlled  by  the  same  organizational 
structure.  Additionally,  the  amount  of  equipment,  materials  and 
overhead  allocated  for  the  exclusive  use  of  the  Civil  Engineering 

Design  Section  is  not  maintained  in  the  records  of  the  Base  Engineer 
Automated  Management  System.  Thus,  including  these  inputs  would  not 
be  keeping  with  the  restraint  to  avoid  an  additional  administrative 
workload. 

Output  measures  are  difficult  to  develop.  The  Civil  Engineering 
Design  Section  has  different  outputs  every  day  of  the  year,  and  one 
design  unit  has  different  outputs  from  the  next.  Output  measures  must 
be  used  which  enable  these  organizations  to  be  compared  to  one 

another.  Kaneda  and  Wallett  came  up  with  twenty-six  possible  output 
measures . 

The  twenty-six  measures  of  Kaneda  and  Wallett  were  used  as  the 

starting  point  in  developing  the  comprehensive  list  of  outputs.  The 

two  panels  were  used  to  help  revalidate  the  output  measures  developed 
and  add  to  or  delete  other  measures.  They  reviewed  the  inputs  and 
outputs  to  ensure  that  the  list  was  complete  (le.  all  resources, 
products,  and  services  were  represented)  based  on  their  experience.  A 
total  of  thirty-three  output  measures  were  considered  and  analyzed. 
From  these,  twenty  measures  were  selected  for  use  in  the  model.  These 
outputs  are: 
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Total  contract  funds  obligated 


Estimated  dollar  amount  of  all  projects  designed 
(complete  and  ready  for  contracting  action) 


Total  O&M  maintenance  and  repair  project  funds 
obligated 


Total  O&M  minor  construction  funds  obligated 


Total  number  of  projects  designed  (complete  and 
ready  for  contracting  action) 


Total  number  of  facility  inspections  and  utility 
system  surveys  completed 


Total  number  of  special  technical  studies  and 
reports  completed 


One  over  total  funds  expensed  on  contract  change 
orders 


One  over  number  of  contract  change  orders 


Total  estimated  dollar  amount  of  in-house  work 
orders  designed 


Total  estimated  dollar  amount  of  architect-engineer 
packages  prepared 


Total  A-E  design  funds  obligated 


Estimated  dollar  amount  of  MCP  Project  Books 


Number  of  work  orders  reviewed  and/or  evaluated 


Number  of  technical  reviews  accomplished  on 
designed  projects 


Pages  of  project  specifications 


Total  number  of  oral  presentations  made 


Number  of  facility  surveys  conducted 


Total  hours  or  surveys  completed 


Number  of  pages  of  engineering  drawings  completed 


The  complete  list  of  inputs  and  outputs  analyzed,  along  with  the 


reason  for  the  rejection  of  some  of  the  measures,  is  presented  in 


Appendix  C.  The  selected  inputs  and  outputs  are  also  further 
explained  in  Appendix  D. 

Several  inputs  and  outputs  have  the  form  of  "one  over”  a 
measure.  This  is  necessary  to  maintain  the  relationship  that  an 
increase  in  inputs  will  result  in  an  increase  in  output.  This  is  the 
positive  relation  referred  to  by  Bessent,  Bessent,  and  Clark  (5). 

Each  input  and  output  measure,  while  being  analyzed,  was 
evaluated  against  several  different  sets  of  criteria.  This  step  was 
necessary  due  to  the  unique  nature  of  the  data  (as  explained  in  the 
following  section,  the  data  was  generated  by  the  authors).  Inspection 
or  statistical  analysis  of  the  data  would  not  reveal  faulty 
measurements,  due  to  its  special  development.  One  of  the  sets  of 
criteria  used  was  deve.loped  In  a  thesis  by  Armstrong  and  Dougherty, 
A  Study  of  the  Development  of  Output  Measures.  The  criteria  are: 

1.  Based  on  determined  mission 

2.  Measure  permits  direct,  evaluation 

3.  Measure  is  based  on  objectives 

4.  Causative  relationship  exists 

5.  Measure  is  needed 

6.  Measurement  is  not  a  restatement  of  resources 

7.  Data  for  measurement  is  quantif iable-programable 

8.  Measurement  is  meaningful  to  management 

9.  Data  for  measure  is  available 

10.  Structure  agreement  exists 

11.  Measurement  is  homogeneous  with  like  measures 

12.  Measure  is  matchable  to  expense 

13.  Measure  provides  for  continuity  and  compatahility 
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14.  Measure  is  not  a  composite  if  other  alternative 
exists  [1:98] 

The  final  list  of  selected  inputs  and  outputs  were  then  evaluated 
against  the  seven  characteristics  of  productivity  measures  recommended 
by  Tuttle  (35:77-78)  and  discussed  in  Chapter  II  of  this  research. 
The  outputs  and  inputs  satisfy  five  of  the  seven  characteristics.  The 
characteristics  satisfied  are  those  of  completeness,  comparability, 
input  coverage,  comparability  with  existing  data  sources,  and 
consistency  across  organizations.  The  two  characteristics  nor 
satisfied  are  cost  effectiveness,  and  acceptable  to  organization 
members.  These  characteristics  can  not  be  appropriately  determined 
until  the  measurement  model  is  implemented  and  In  use.  Only  at  thin 
time  can  the  final  two  characteristics  be  determined.  With  five  of 
Tuttle's  seven  characteristics  satisfied,  and  Bessent,  Bessent,  and 
Clark's  (5)  rules  for  inputs  and  outputs  complied  with,  the  selected 
inputs  and  outputs  were  used  for  data  generation. 

Data  Generation  and  Preparation  for  Analysis 

The  data  used  in  the  development  of  the  productivity  measuremern 
model  was  generated  by  the  researchers.  Actual  data  from  real  Civil 
Engineering  Design  Sections  was  not  gathered. 

The  decision  to  simulate  real  data  is  based  primarily  on  th  ■ 
choice  of  the  statistical  analysis  method.  Recent  research  ha 
produced  a  new  method  for  measuring  the  relative  efficiency  of  an 
organization  with  respect  to  other  similar  organizations  performing 
similar  operations  with  similar  resources.  That  method  is  called 


Constrained  Facet  Analysis  (CFA)  and  is  explained  in  detail  in  this 
chapter  in  the  discussion  of  analysis  methods. 

The  authors  believe  that  Constrained  Facet  Analysis  will  prove  to 

be  a  rowerful  management  tool  when  knowledge  of  its  existence  spreads 

and  more  applications  of  the  model  are  made.  The  authors  have  no 

reason  to  believe  that  the  model  cannot  successfully  be  implemented  in 

an  Air  Force  Civil  Engineering  Design  Section.  However,  it  is 

recognized  that.  CFA  has  not  yet  been  applied  to  a  civil  engineering 

problem.  Any  application  of  the  model  to  Civil  Engineering  using 

actual  data  could  not  be  reliably  shown  to  reflect  the  true  state  of 

nature,  unless  the  true  state  of  nature  was  already  known.  Therefore, 

the  use  of  CFA  in  a  civil  engineering  application,  such  as  a  design 

section,  Is  needed  to  demonstrate  its  capability,,  but  it  must  be 

applied  so  that  the  output  of  the  model  verifies  a  previously  known 

* 

state  of  nature.  One  reliable  way  of  knowing  the  true  state  of  nature 
is  to  manufacture  It. 

Data  Generation.  Identification  of  inputs  and  outputs 
revealed  26  candidate  measures.  As  stated  in  the  analysis  section, 
data  from  twice  as  many  organizations,  or  52  units,  will  be  used  in 
the  analysis. 

The  data  will  reflect  the  following  situations: 

])  Efficient  large  units  with  a  concentration  of 
in-house  design  work 

2)  Inefficient  large  units  with  a  concentration 
of  in-house  design  work. 

3)  Efficient  small  units  with  a  concentration  of 
in-house  design  work. 

4)  Inefficient  small  units  with  a  concentration  of 
in-house  design  work. 
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5)  Efficient  large  units  with  design  work 
contracted  out  to  A-E  firms. 

6)  Inefficient  large  units  with  design  work 
contracted  out  to  A-E  firms. 

7)  Efficient  small  units  with  design  work 
contracted  out  to  A-E  firms. 

8)  Inefficient  small  units  with  design  work 
contracted  out  to  A-E  firms. 

Based  on  the  authors'  experience,  some  typical  values  for  each 
input  and  output  were  determined.  Using  that  group  of  values,  two 
ranges  were  developed— one  for  small  organizations  and  one  f>  .  large 
organizations.  Within  each  of  those  two  ranges,  values  were  then 
adjusted  to  reflect  either  an  in-house  design  concentration  or  an  A-E 
contract  design  concentration.  Thus,  four  major  divisions  were 
developed  within  the  organizations.  These  divisions  are  1)  large 
units  with  in-house  design  work,  2)  large  units  with  A-E  contracted 
design,  3)  small  units  with  in-house  design  work,  4)  small  units  with 
A-E  contracted  design.  One  organization  in  each  division  was  chosen 
to  be  the  efficient  organization  and  one  organization  selected  to  be 
the  least  efficient  organization.  The  most  efficient  organization  was 
then  assigned  the  lowest  input  values  within  its  division  and  the 
highest  output  values  for  that  division.  The  organization  selected  to 
be  least  efficient  was  given  the  highest  inputs  and  the  lowest  output 
values  for  its  division.  By  maintaining  the  values  of  inputs  and 
outputs  within  the  prescribed  divisions,  it  was  hoped  that  the 
particular  "neighborhood"  of  organizations  could  be  fixed.  The  data 
was  generated  for  the  following  conditions  to  exist: 


,  iiv, 
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1)  Organization  1,  the  efficient  large  unit  with 
a  concentration  of  in-house  design  work. 

2)  Organization  10,  the  least  efficient  large  unit 
with  a  concentration  of  in-house  design  work. 

3)  Organization  12,  the  efficient  small  unit  with  a 
concentration  of  in-house  design  work. 

4)  Organization  23,  the  least  efficient  small  unit 
with  a  concentration  of  in-house  design  work. 

5)  Organization  24,  the  efficient  large  unit  with 
design  work  contracted  out  to  A-E  firms. 

6)  Organization  32,  the  least  efficient  large  unit 
with  design  work  contracted  out  to  A-E  firms. 

7)  Organization  35,  the  efficient  small  unit  with 
design  work  contracted  out  to  A-E  firms. 

S)  Organization  46,  the  least  efficient  Gmail  unit 
with  design  work  contracted  out  to  A-E  firms. 

Preparation  of  Data  for  Analysis.  The  computer  code  for  CFA 
was  developed  and  tested  at  the  University  of  Texas  at  Austin,  and  is 
maintained  on  that  school's  CDC  Dual  Cyber  170/750.  The.  code  is 
written  in  Fortran  and  employs  a  revised  primal  simplex  code  (7:1). 
The  authors  traveled  to  the  University  of  Texas  at  Austin  and  were 
permitted  to  use  the  Cyber  and  the  CFA  Code  in  the  analysis  of  their 
data. 

Use  of  the  CFA  code  required  preparation  of  the  data  in  the 

proper  format,  as  outlined  in  the  paper  by  Bessent,  Bessent,  Clark, 
and  Elam  describing  the  code  (7:6).  The  data  from  each  of  the 
fifty-two  units  was  entered  in  the  Aeronautical  Systems  Division  (ASD) 
Cyber  computer  at  Wright-Pa tterson  Air  Force  Base,  Ohio.  A  hard-copy 
of  the  data  file  was  printed  and  all  data  entries  were  checked  for 
accuracy  against  the  original  data  list.  The  data  file  was  then 
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routed  to  a  card  printer,  where  the  data  was  punched  onto  computer 
cards  (four  cards  per  organization).  The  cards  were  then  entered  into 
a  card  reader  to  verify  that  the  data  items  had  been  punched 
accurately. 

Upon  arrival  at  the  University  of  Texas  at  Austin,  the 

researchers  were  given  a  briefing  on  the  use  of  the  school's  Cyber 
computer  and  the  CFA  code.  The  data  card  deck  and  appropriate  control 
cards  were  entered  Into  the  computer  to  establish  a  Iccal  data  file. 
Some  data  cards  could  ot  be  read  properly  by  the  school's  card 
reader,  requiring  some  additional  manual  data  input.  A  copy  of  the 
data  file  is  contained  in  Appendix  E. 


IV.  Findings  and  Results 
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The  data  file  In  Appendix  E  was  analyzed  using  the  CFA  code 
contained  in  the  University  of  Texas  at  Austin's  Dual  Cyber  computer. 
The  analysis  runs  were  made  the  week  of  25-28  June,  1984. 

After  entry  of  the  execution  command  for  the  CFA  code,  the  code 
instructs  the  user  to  specify  requirements  of  the  analysis  from  five 
options.  First,  the  code  asks  which  organizational  units  are  to  be 
included  in  the  reference  set  to  be  analyzed.  The  authors  requested 
all  organizations  be  included  in  each  run.  Second,  the  code  asks 
which  organizations  of  the  reference  set  are  to  be  processed.  Again, 
we  specified  all  organizations  be  processed.  The  next  two  options 
ask  which  outputs  and  which  inputs,  respectively,  are  to  be  included 
in  the  analysis.  Finally,  the  code  ask3  which  of  the  managerial 
reports,  if  any,  are  to  be  produced  in  the  analysis.  After  the  five 
options  are  speclfleJ,  che  program  executes. 

Initial  runs  on  the  data  set  were  unsuccessful.  Consultations 
with  Dr.  A.  Bessent  and  Dr.  W.  Bessent,  Co-Directors  of  the 
Educational  Productivity  Council  (which  supports  the  research  and 
maintenance  of  the  CFA  code),  revealed  that  the  CFA  code  was 
dimensioned  such  that  a  combination  of  only  twenty  (20)  inputs  and 
outputs  could  be  used.  Dr.  A.  Bessent  attempted  to  redimension  Che 
code  so  that  up  to  100  inputs  and  outputs  could  be  analyzed,  but  her 
efforts  were  unsuccessful. 

Since  one  of  the  options  of  the  code  is  to  select  which  inputs 
and  outputs  are  to  be  considered,  the  authors  then  selected  a  total 
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of  twenty  of  the  inputs  and  outputs  for  analysis.  Again,  the 
computer  computation  was  aborted.  Subsequent  tests  with  succeedingiy 
smaller  combinations  of  inputs  and  outputs  resulted  in  a  successful 
run  when  a  total  of  fifteen  factors  were  considered. 


Inputs  and  Outputs  Used 

Originally  both  the  productivity  of  design  engineers  and  the 
site  development  personnel  were  to  be  included  in  the  measurement 
model.  However,  limitations  of  the  computerized  model  required  a 
reduction  in  the  inputs  and  outputs.  The  site  development  personnel 
were  eliminated  from  both  the  inputs  and  outputs.  This  resulted  in 
total  elimination  of  the  following  outputs: 

-Number  of  surveys  conducted. 

-Total  hours  of  surveys  completed. 


The  list  of  inputs  and  outputs  were  still  too  numerous  for  the 
computerized  version  of  CFA  at  the  University  of  Texas.  To  eliminate 
inputs  and  outputs  further  it  was  determined  to  begin  with  those  that 
have  the  smallest  percentage  cf  manhours  devoted  to  them.  To 
determine  this  the  authors  used  data  obtained  from  the  AFIT  School  of 
Civil  Engineering  and  presented  in  the  School's  Engineering  and 
Environmental  Planning  Management  Applications  Course.  This  method 
resulted  in  the  elimination  of  the  following  inputs: 

-Number  of  professional  education  courses  completed. 

-One  over  the  number  of  additional  duties  performed. 
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and  the  following  outputs: 


-Total  O&M  minor  construction  funds  obligated. 

-Total  number  of  facility  inspections  and  utility 
system  surveys  completed. 

-Total  number  of  special  technical  studies  and 
reports  completed. 

-Pages  of  project  specifications. 

-Total  number  of  oral  presentations  made. 

-Number  of  pages  of  engineering  drawings  completed. 


The  final  output  eliminated  was  "Total  A-E  design  funds  obligated." 
It  was  eliminated  because  of  indecision  on  the  part  of  the  authors 
whether  it  was  truly  an  output  or  an  input.  Rather  than  jeapordize 
the  analysis  by  including  it  as  an  output,  it  was  eliminated 
entirely.  This  was  possible  since  the  A-E  contrac.t  output  portion  is 
captured  in  the  output, "Total  estimated  dollar  amount  of 

architect-engineer  packages  prepared." 

That  reduction  left  a  total  of  four  inputs  and  eleven  outputs. 
The  GFA  model  was  able  to  accommodate  that  number  of  factors.  These 
inputs  and  outputs,  then,  were  used  in  the  final  analysis: 

Inputs 


1. 

Labor 

manhours 

2 , 

Labor 

cos  ts 

3. 

Years 

experience 

4. 

Personnel  skill  level 

Outputs 

1.  Total  contract  funds  obligated 
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2.  Estimated  dollar  amount  of  all  projects  designed 
(complete  and  ready  for  contracting  action) 

3.  Total  O&M  maintenance  and  repair  project  funds 
obligated 

4.  Total  number  of  projects  designed  (complete  and 
ready  for  contracting  action) 

5.  One  over  total  funds  expensed  on  contract  change 
orders 

6.  One  over  number  of  contract  change  orders 

7.  Total  estimated  dollar  amount  of  in-house  work 
orders  designed 

8.  Total  estimated  dollar  amount  of 
architect-engineer  packages  prepared 

9.  Estimated  dollar  amount  of  MCP  Project  Books 

10.  Number  of  work  orders  reviewed  and/or  evaluated 

11.  Number  of  technical  reviews  accomplished  on 
designed  projects 


Model  Execution 

The  computerized  model  was  run  with  all  52  organizations  and  the 
4  inputs  and  11  outputs.  The  following  results  were  obtained: 


1.  Organization  1,  the  efficient  large  unit  with 
a  concentration  of  in-house  design  work. 

2.  Organization  10,  the  least  efficient  large  unit 
with  a  concentration  of  in-house  design  work. 

3.  Organization  12,  the  efficient  small  unit  with 
a  concentration  of  in-house  design  work. 

4.  Organization  23,  the  least  efficient  small  unit 
with  a  concentration  of  in-house  design  work. 

5.  Organization  24,  the  efficient  large  unit  with 
design  work  contracted  out  to  A-E  firms. 
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6.  Organization  32,  the  least  efficient  large  unit 
with  design  work  contracted  out  to  A-E  firms. 

7.  Organization  35,  the  efficient  small  unit  with 
design  work  contracted  out  to  A-E  firms. 

8.  Organization  46,  the  least  efficient  small  unit 
with  design  work  contracted  out  to  A-E  firms. 

This  coincides  with  the  results  anticipated  in  the  methodology 
chapter  of  this  research.  The  efficiencies  of  all  52  organizations 
are  listed  in  Appendix  F.  The  actual  computer  printout  results  are 
included  in  Appendix  G. 

The  model  gives  two  efficiencies.  These  are  termed  as  the  upper 
and  lower  bound  efficiencies.  The  upper  bound  is  the  largest 
relative  efficiency  possible  for  that  organization  and  the  lower 
bound  is  the  smallest  possible  relative  efficiency  for  that 
organization.  The  actual  efficiency  may  lay  anywhere  within  these 
two  bounds. 

The  indication  of  only  four  efficient  organizations  would  lead 
us  to  believe  that  in  fact  four  "neighborhoods"  or  divisions  were 
used  by  the  model  as  was  desired.  This  means  that  small  in-house 
design  organizations  were  only  evaluated  against  small  in-house 
design  organizations,  etc. 

The  model  also  gives  reports  for  each  inefficient  organization 
showing  what  inputs  would  be  necessary  for  that  organization  to  be 
efficient  if  it  maintained  the  same  output.  The  model  also 
determines  what  output  increases  are  necessary  for  an  organization  to 
become  efficient  if  its  present  inputs  remain  the  same.  A  sample  of 
each  report  is  included  in  Appendix  H. 
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Limitations  of  the  CFA  Code 

Use  of  the  CFA  code  as  It  presently  exists  revealed  some 
limitations  which  anyone  interested  in  application  of  the  code  or 
further  research  in  this  area  should  take  note  of.  Improvements  to 
the  CFA  code  are  being  made  continuously,  so  some  of  the  following 
problems  may  not  be  of  concern  in  the  near  future. 

In  the  development  of  the  code,  the  case  of  a  set  of  inputs  and 
outputs  exceeding  twenty  in  number  was  unforeseen.  The  CFA  code  will 
not  work  for  a  case  with  more  than  twenty  inputs  and  outputs. 
Depending  on  the  data  and  its  range,  the  code  may  not  work  with  more 
than  fifteen  factors  involved.  Work  is  underway  to  expand  the 
capability  of  the  code  in  this  regard.  '  Until  the  problem  is 
resolved,  researchers  are  forced  to  minimise  the  number  of  inputs  and 
outputs  used.  Pearson's  correlation  can  be  used  on  actual  data  to 
discard  those  factors  having  a  high  positive  correlation  (meaning  one 
of  the  factors  is  sufficient  to  measure  the  activity)  or  those  having 
a  negative  correlation  (meaning  additional  amounts  of  that  factor 
detract  from  productivi ty) .  Pearson's  correlation  was  not  used  in 
this  research  effort  to  reduce  the  set  of  inputs  and  outputs  because 
the  data  was  generated  by  the  researchers.  Any  highly  positive  or 
aegative  correlations  would  be  suspect. 

Related  to  the  above  problem,  it  may  be  advantageous  in  the 
description  of  an  organization's  productivity  to  first  focus  on  a  few 
main  inputs  and  outputs  in  the  analysis.  After  successful  analyses 
have  been  made  of  those  few  factors,  the  researcher  can  expand  and 
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become  more  descriptive  with  each  input  and  output.  Additionally, 
select  only  those  inputs  and  outputs  having  a  positive  correlation. 
While  factors  negatively  correlated  with  productivity  can  be 
converted  by  using  reciprocals  or  inverses,  these  manipulations  tend 
to  cause  computation  problems  during  program  execution  by  generating 
extremely  small  numbers. 
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V.  Conclusions  and  Recommendations 


Conclusions 

The  measurement  of  productivity  in  an  Air  Force  Civil  Engineering 
Design  Section  is  desirable  from  a  management  perspective.  Until  now, 
inadequate  measurement  techniques  prevented  managers  from  knowing  how 
well  their  units  were  performing  over  time  or  with  respect  to  other 
units.  The  use  of  the  Constrained  Facet  Analysis  technique  can 
provide  that  information.  CFA  is  a  valid  method  of  determining  the 
relative  efficiency  of  a  base  level  Civil  Engineering  Design  Section. 

This  research  effort  is  a  step  forward  in  the  DOD  and  Air  Force 
Productivity  Programs.  Further  research  is  needed  to  develop  CFA's 
full  potential  of  helping  managers  better  understand  productivity  and 
improve  organizational  performance. 

t 

Research  Questions  Answered 

Three  research  questions  were  presented  in  Chapter  I  of  this 
thesis.  In  this  section,  each  of  the  three  questions  are  reiterated 
along  with  a  brief  summation  of  their  respective  answers. 

The  first  research  question  posed  in  this  thesis  was,  "What  are 
the  inputs  and  outputs  for  a  base  level  Civil  Engineering  Design 
Section?"  The  total  list  of  twenty  outputs  and  six  inputs  described 
in  the  Methodology  chapter  of  this  research  are  the  inputs  and  outputs 
for  a  Design  Section,  The  complete  list  could  not  be  used  in  the 
Constrained  Facet  Analysis  technique  used  by  the  researchers  for  the 
development  of  the  model,  however,  due  tc  computer  limitations.  A 


reduction  of  relatively  insignificant  and/or  redundant  measures  was 
made  to  meet  the  computer  requirements. 

The  second  research  question  asked,  "What  is  the  appropriate 
productivity  measurement  technique?”  Three  different  analysis  methods 
were  examined,  after  which  Constrained  Facet  Analysis  was  chosen.  CFA 
is  the  only  technique  capable  of  handling  multiple  inputs  and  outputs 
and  producing  a  single  aggregate  measure  relative  to  a  frontier  of 
actual  achievement. 

The  final  research  question  asked,  "Can  a  model  of  productivity 
be  developed  using  appropriate  inputs  and  outputs?"  A  model  was 
developed,  using  the  framework  of  the  CFA  model  adapted  to  a  Civil 
Engineering  design  context  through  the  use  of  the  Design  Section 
inputs  and  outputs.  Based  on  the  results  of  the  data  analysis 
(presented  in  Chapter  IV),  the  adaptation  yielded  a  valid  productivity 
measurement  model. 

Recommendations 

The  authors  recommend  that  further  research  be  done  before  any 
attempt  is  made  to  implement  CFA  into  design  sections.  Actual  data 
should  be  collected  from  the  Continental  United  States  (CONUS)  Air 
Force  bases.  Pearson's  correlations  should  be  conducted  for  each 
input  and  output  to  screen  out  those  factors  that  are  redundant 
measurements  or  negatively  correlated.  The  data  should  then  be 
analyzed  using  CFA.  Assuming  that  further  research  supports  the 
notion  that  valid  productivity  measurement  within  design  sections  is 
possible,  then  we  recommend  implementation  of  the  model. 
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We  recommend  that,  should  this  analysis  technique  be  implements J , 
design  section  managers  be  trained  in  its  use  and  application,  and  in 
its  limitations.  The  managers  should  be  shown  how  it  can  help  tln-m 
improve  their  organisation's  performance,  rather  than  becoming  another 
mandated  program  that  impedes  their  progress. 

We  recommend  that  any  application  of  the  CFA  technique  be  dune 
initially  as  an  aid  to  the  base  level  managers,  rather  than  as  an 
organizational  assessment  tool  used  at  the  Major  Command  level . 
Results  of  the  analysis  should  be  given  to  the  base  level  managers  •  or 
their  use  in  effecting  change  to  improve  their  organization.  The 
analysis  results  would  include  the  overall  efficiency  rating  of  'he 
particular  organization,  summary  reports  of  which  inputs  to  decrease 
or  outputs  to  increase  to  become  efficient,  and  a  list  of  the 
organizations  against  which  they  were  compared.  Armed  with  that 
information,  the  manager  could  then  change  those  factors  over  which  he 
has  control  to  move  his  organization  closer  to  the  efficiency 
frontier.  He  can  contact  those  units  within  the  frontier 
"neighborhood"  to  see  what  those  organizations  are  doing  that  makes 
them  efficient.  Additionally,  he  can  identify  those  factors  beyund 
his  control  which  are  contributing  to  his  inefficiency  rating,  and 
possibly  use  the  ratings  to  induce  upper  management  to  change  thu^o 
inputs  and  outputs  in  his  favor. 

As  with  any  computer-generated  product,  the  results  are  only  as 
good  as  the  information  used.  As  numbers  cannot  totally  desribt-  a 
situation,  managerial  common  sense  must  enter  into  the  evaluation. 
Each  inefficiency  rating  requires  investigation  into  the  circumstan> 


surrounding  that  organization's  poor  score.  An  inefficient  rating  can 
identify  an  organization  needing  improvement,  but  that  rating  may  not 
be  a  valid  assessment  of  the  manager's  efforts  or  problems.  A  poor 
manager  can  Inherit  an  efficient  organization,  just  as  an  excellent 
manager  can  find  himself  in  an  extremely  inefficient  unit.  While 
measurements  over  time  could  yield  information  on  a  given  manager's 
performance,  a  single  evaluation  cannot.  Therefore,  we  would  not 
recommend  that  the  results  of  the  model  be  used  as  an  evaluation 


technique  for  managers. 
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Appendix  A:  CFA  Model 


Model  Used  in  Constrained  Facet  Analysis 
of  Not-Fully-Enveloped  Units  [8:16-19] 

The  model  used  in  the  iterative  method  called 
Constrained  Facet  Analysis  is  presented  in  this  appendix, 
a  model  which  can  be  used  in  evaluating  the  range  of 
inefficiency  in  organizational  units  and  in  determining 
marginal  rates  of  substitution  and  productivity  in 
frontier  facets. 

Suppose  one  wishes  to  evaluate  the  relative 
efficiency  of  n  decision  making  units  (DMUs),  each  of 

which  uses  varying  amounts  of  m  inputs  and  produces 

* 

varying  amount  of  s  outputs.  Using  notation  conventions 
similar  to  those  used  by  Clark  (see  references  at  the  end 
of  this  appendix),  let: 

xii  “  the  amounts  of  input  type  i  used  by  DMU  j 

during  the  period  of  observation,  i  «*  1,2,.  ..,ra 
and  j  =  1,2,. . . , n . 

yrj  ■  the  amount  of  output  type  r  produced  by  DMU  j 

during  the  period  of  observation,  r  =  1,2 . s 

and  j  ■  1,2,... , n . 

xik  *  the  amount  of  input  type  i  used  by  the  unit  k 

where  k  j  =  {1,2 . k,...,n}  and  unit  k  is 

the  DMU  being  evaluated.  Each  DMU  in  turn  will 
be  evaluated. 

yrk  *  amount  of  output  type  r  used  by  DMUj.. 

N  =  1,2,3,... ,M  is  the  sequence  of  iterations 

of  the  Constrained  Facet  Analysis  model  which 
ends  at  iteration  M. 
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-  the  upper  bound  efficiency  value  sought  for  DMUk 
which  is  determined  from  the  solution  of  the  first 
iteration  of  the  Constrained  Facet  Analysis. 

^  «  the  lower  bound  efficiency  value  sought 

for  DMUk  which  is  determined  from  the  solution 
of  the  final  iteration  (M)  of  the  Constrained 
Face  t  Analysis  . 

vlk<»>  ■  the  multipliers  for  each  input  type  1 

which  will  be  determined  by  solution  of  the  Nth 
iterative  model. 

U  «  the  multipliers  for  each  output  type  r 

which  will  be  determined  by  solution  of  the  Nth 
iterative  model. 


srk 


(N-l)*a 


the  dual  surplus  values  associated  with 

outputs  r  »  1,2 . s  of  DMUk  at  optimality 

of  the  previous  iteration.  For  -the  initial 
iteration,  these  surplus  values  are 


( N- 1 )  * 


(0)* 


yrlt- 


the  dual  surplus  values  associated  with 
inputs  i  ■  1,2,.  .  .  ,m  at  optimality  of  i  .ie 
previous  iteration.  Initial  values  at  iteration 
one  are 


^The  form  of  the  CFA  model  used  in  the  first  iteration 
is  similar  to  the  Data  Envelopment  Analysis  (DEA)  model  of 
Charnes,  Cooper  and  Rhodes  [1],  however,  the  non-Archimedean 
infinitesimal  quantities  are  not  required. 
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The  following  linear  programming  model  is  used  in 
constrained  facet  analysis  for  each  iteration 

N  =  1 ,  2, . . . .M: 


■  -V  *. 
,T  I,"  % 


4 

!i.r 


Primal 


m 


Ma*  fk‘»’  -  I  *  I  ''ik(N,nk‘"-l>*  ID 

r=l  i=l 


s.t. 


s  m 

l  ta(M>ycj-  I  vik(N1xi3-0  *°i3c%<N) 

r=l  i=l 


IT 


•v 


l'K.V 


*  .  • 
•  ^  ‘  w  ", 

*>  A 

M 

,<r  JP 


a 


.  4\ 
A*A 


s  m  _ 

I  ^rk(N)V;;j  -  I  vik(N)xij  -  0  for  jeEk 


(N) 


11=1 


i=l 


m 


I  Vik(N)xik  =  1 
i=l 


Mrk(NJ,  vik(W)  >  0 


where 
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t j/jth  constraint  holds  with  equality  at 
optimality  at  iteration  N-1  } 


Ek(N) 

Ek(N) 

Ek ^ 1  ^  =  <i>  (empty),  t 1  )  =  {1,  2,..,,n) 


{j/jth  constraint  is  <  0  at  optimality  of 
iteration  N-1 } 
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The  upper  and  lower  bound  efficiency  measures  are 


obtained  from  solution  of  the  first  and  last  iterative 


models  as  shown  below: 


hk(1)  -  £k(1>  -1=1  Urk(1,*yrk 

r=1 


hk(M)  =  1  hrk(M)*yrk 

r=1 


The  dual  of  model  (1)  above  is: 


Dual 


Min  Uj^N) 


I,.,  *  lIM  r,ai*r,  ■  ■*' 


(N~l ) * 


r  »  1 , 2 ,  .  .  . ,  s 


(N)  _  Y 


l  j  L  Yj 


(N ) v  .  . 

-ID 


S  ,(N) 
aik 


=  =.,  (N-D* 
5ik 


l  m 


unrestricted;  srk^N1,  >  o 


The  mathematical  theory  and  proofs  related  to  the 
development  of  this  model  can  be  found  in  Clark  [2]  and 
will  not  be  repeated  in  this  appendix.  But  there  are  a 
few  model  characteristics  which  are  worth  noting  here. 

First,  the  efficiency  measures  h^1'^'1  and  h^^ 
are  scalar  ratio  measures.  Secondly,  the  constraints 
of  the  primal  problem  insure  that  the  maximum  achievable 
value  of  these  efficiency  measures  is  1.  Furthermore, 
Constrained.  Facet  Analysis  does  not  require  that  outputs 
or  inputs  have  common  scales  or  units  of  measurement,  an 
important  attribute  when  dealing  with  difficulties  such 
as  nonmonetary  objectives  and  nonpurchased  resources. 
However,  all  measured  input  and  output  values  are 

required  to  be  strictly  positive.  Finally,  each  unit  is 
compared  to  others  in  the  set  which  have  similar 

Input/output  mixes,  i.e.,  those  units  in  its 

"neighborhood  .  “ 

In  short,  the  Constrained  Facet  Analysis  model  can 
identify  units  which  are  efficient:  or  inefficient 
relative  to  a  neighborhood  frontier  region  of  actual 
achievement;  it  can  provide  a  limited  number  of  cluas  on 
possible  causes  from  analysis  of  surplus  variables  and 

multipliers;  and  it  is  helpful  in  evaluating  the  impact 
of  alternative  mixes  of  inputs  and  outputs. 

Furthermore,  the  information  provided  by  the  CFA 

model  is  a  major  Improvement  over  the  inadequate,  partial 
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(and  sometimes  inaccurate)  measures  of  performance  which 


are  now  typically  in  use  in  many  public  service 
organizations.  In  addition  to  its  usefulness  as  a 
performance  monitoring  device,  this  efficiency  analysis 
tool  opens  the  door  for  further  development  and  growth  in 
other  areas  of  planning,  resource  allocation  and  decision 
support. 
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Appendix  B:  Panel  Members 


Panel  of  AFIT  School  of  Systems  and  Logistics 
Engineering  Management  Students 

NAME/RANK  CUM  YEARS  IN  BCE 

James  T.  Ryburn,  Captain  9 

Frederick  Nightengale,  Captain  6 

Charles  Howell,  Captain  8 


Panel  of  AFIT  School  of  Civil  Engineering 

"  Facul ty 


NAME/ RANK 

Jeffery  R.  Charles,  Captain 
Thomas  H.  Gross,  Major 


POSITION 

Course  Director,  Dept 
of  Management  Applications 

Course  Director,  Dept 
of  Management  Applications 
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Appendix  C:  List  of  Inputs  and  Outputs  Analyzed 


OUTPUT  MEASURES 

1.  Total  estimated  dollar 
amount  of  contract  projects 
and  in-house  work  orders 
designed 

2.  Total  contract  funds 
obligated 

3.  Total  O&M  contract  funds 
obligated 

4.  Total  O&M  maintenance  and 
repair  funds  obligated 

5.  Total  O&M  minor  construction 
funds  obligated 

6.  Total  estimated  dollar 
amount  of  project 
document  (DD  1391/1391C) 
completed 

7.  Total  number  of  projects 
designed  (complete  and 
ready  for  acquisition  action) 

8.  Total  number  of  facility 
inspections  and  utility 
system  surveys  completed 

9.  Total  number  of  special 
technical  studies  and 
reports  completed 

10.  One  over  total  funds  expensed 
on  contract  change  orders 

11.  One  over  number  of  contract 
change  orders 

12.  Total  estimated  dollar  amount 
of  in-house  work  orders 
designed 

13.  Total  service  contract  funds 
obligated 


REASON  FOR  ELIMINATION 

This  is  the  addition  of 
output  measures  2  and  12 


DD  1 39 1 / 139 1 C  are  the 
responsibili-ty  of  DEEV 


Service  contracts  are  handled 
by  DEEC 
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14.  Number  of  environmental 
assessments  (EA's)  and 
environmental  impact 
statements  (EIS's) 
completed 

15.  Total  estimated  dollar 
amount  of  architect- 
engineer  packages 
prepared 

16.  Total  A-E  design  funds 
obligated 

17.  Number  of  work  orders 
reviewed  and/or  evaluated 

18.  Number  of  technical  reviews 
accomplished  on  designed 
projects 

19.  Number  of  military  family 
housing  (MFH)  inspections 
completed 


20.  Pages  of  project 
specifications 

21.  Total  hours  of  supplemental 
training  completed 


22.  Number  of  professional 
educational  courses 
completed 

23.  Total  hours  construction 
inspections  completed 


While  assistance  may  be  given, 
EA's  and  EiS's  are 
accomplished  by  DEEV 


To  avoid  counting  the 
inspections  accomplished  by 
the  Housing  Office,  those 
completed  by  DEEE  will  be 
added  in  output  8, 


Evaluated  as  both  an  input 
and  an  output  and  not 
considered  to  be  a 
significant  amount  at  a  base 
level  organization 

Was  originally  evaluated  as 
an  output,  but  in  the  opinion 
of  the  researchers  should  be 
included  as  an  input  instead 

Construction  inspections  are 
handeled  by  DEEC.  Those  that 
are  completed  by  DEEE  will 
be  included  in  output  8 


24.  Total  number  of  oral 
presentations  made 


25.  Total  number  of  journal 
articles  written 


26.  Hours  of  training  sessions 
taught/conducted 

27.  Number  of  surveys 
conducted 

28.  Total  hours  of  surveys 
completed 

29.  Number  of  pages  of 
engineering  drawings 
(blueprints)  completed 

30.  Customer  satisfaction  index 


31.  Number  of  buildings 
demolished 


32.  Estimated  dollar  amount 
of  all  projects  designed 
(completed  and  ready  for 
acquisition  action) 

33.  Estimated  dollar  amount  of 
MCP  Project  Books 


Not  part  of  military  job.  If 
accomplished,  should  be 
completed  on  individual  time 

Not  considered  to  be  a 
significant  amount  in  a 
base  level  organization 


Time  lag  between  facility 
design  and  customer  use  is 
too  long 

Engineers  are  not  always 
involved.  Could  be  performed 
on  a  service  contract  handled 
strictly  by  DEEC 
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INPUT  MEASURES 


REASON  FOR  ELIMINATION 


1 .  Labor  manhours 

2.  Labor  costs 

3.  Years  experience 

4.  Personnel  skill  level 
aggregate 

5.  Number  of  professional 
education  courses  completed 

6.  One  over  additional  duties 
performed 
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Appendix  D:  Inputs  and  Outputs  Further  Defined 


All  outputs  and  inputs  are  cumulative  values  for  all  personnel  in 
design  (those  figured  into  the  manhours  calculation)  for  the  fiscal 
year.  Data  could  be  broken  down  into  quarterly  figures  and  used  in 
the  productivity  model,  if  desired. 

OUTPUTS  FURTHER  DEFINED 


1.  Total  contract  funds  obligated-  The  total  amount  of  design 
contract  funds  obligated.  Outputs  3,  4,  and  10  are  subsets  of  output 

1.  Should  also  include  Military  Construction  Program  (MCP)  and 
Military  Family  Housing. 

2.  Estimated  dollar  amount  of  all  projects  designed-  All  projects 
designed  during  the  year  that  are  complete  and  ready  for  contracting 
action.  This  output  should  also  measure  the  "on  the  shelf"  effort 
during  the  year. 

3.  Total  O&M  raaintenance/repair  project  funds  obligated-  Total 
maintenance  and  repair  project  funds.  This  should  be  the  majority  of 
work  and  funds. 

4.  Total  0&M  minor  construction  funds  obligated-  Should  be  a  small 


percentage  of  output  1. 


5.  Total  number  of  projects  designed  (complete  and  ready  for 
contracting  action)-  This  should  be  the  number  of  those  projects 
included  in  output  2.  This  measure  of  quantity  will  prevent  one  largr 
project  from  dominating  as  could  be  the  case  in  output  2. 

6.  Total  number  of  facility  inspections,  utility  system  surveys  and 
construction  site  inspections  completed-  This  should  include  the 
yearly  facility  inspections  where  the  engineer  accompanies  a  planner, 
utility  system  surveys  that  the  engineers  may  be  involved  with  and 
also  construction  site  inspections.  The  construction  site  Inspections 
should  be  in  support  of  or  along  with  construction  management. 

* 

7.  Total  number  of  special  technical  studies  and  reports  completed- 
This  should  include  any  cost  savings  studies,  replacement  vs  repair 
studies  etc.  There  i3  a  trend  in  the  Air  Force  toward  increased 
emphasis  in  this  area. 

8.  One  over  total  funds  expensed  on  contract  change  orders-  By  taking 
the  reciprocal  value,  we  are  preventing  an  increase  in  change  order 
funds  to  also  increase  productivity.  This  output,  is  attempting  to 
measure  quality  of  the  designs  accomplished. 

9.  One  over  the  number  of  contract  change  orders-  This  recipocal  is 
to  prevent  one  large  change  order  from  dominating  as  can  he  done  in 
output  8. 
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10.  Total  estimated  dollar  amount  of  in-house  work,  orders  designed- 
This  is  the  estimated  dollar  amount  of  projects  designed  to  be 
completed  by  the  in-house  work  force.  Those  projects  are  accomplished 
by  Civil  Engineering  Operations  Branch  personnel  (DEM). 

11.  Total  estimated  dollar  amount  of  architect-engineer  packages 
prepared-  The  estimated  dollar  amount  of  projects  that  the  engineer 
must  prepare  prior  to  submission  to  an  architect-engineer  firm. 

12.  Total  Architect-Engineer  design  funds  obligated-  The  amount  of 
Title  l-B  design  funds  obligated  to  an  Architect-Engineer  firm. 

13.  Estimated  dollar  amount  of  MCP  Project  Books-  The  estimated  dollar 
amount  of  those  Military  Construcion  Program  Project  Books  prepared  by 
the  design  section.  This  value  may  be  zero  if  MCP  PB's  are  ‘prepared 
by  Envii  'nmental  Planning  Section. 

14.  Number  of  work  orders  reviewed  and/or  evaluated-  The  number  of 
work  orders  that  the  engineers  render  assistance  on,  advise  c c  review 
for  the  DEM  personnel 

15.  Number  of  technical  reviews  accomplished  on  designed  projects- 
’Id  include  the  353!  review,  the  60-65%  review  and  ihe  95%  review. 

u. .  .de  only  those  reviews  actually  accomplished. 
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16.  Pages  of  project  specification-  The  number  of  pages  of 
specifications  written  by  the  design  section  personnel.  Should  not 
include  the  pages  added  by  Base  Contracting. 

17.  Total  number  of  oral  presentations  made-  Should  include 
presentations  such  as  paintplan,  special  command  intercat  projects  and 
presentations  to  MAJCOM  personnel.  Should  ncc  Include  daily  or  weekly 
standups. 

18.  Number  of  surveys  conducted-  The  number  of  site  surveys  conducted 
by  the  site  development  personnel.  Should  also  include  plane  crash 
surveys. 

19.  Total  hours  of  surveys  completed-  The  manhours  expended  in 

ft 

performing  the  surveys  listed  in  output  18.  This  measure  is  to  help 
take  into  account  a  large  plane  crash  survey  that  might  be 
accomplished. 

20.  Number  of  pages  engineering  drawings  completed-  Those  drawings 
done  by  the  site  development  personnel  and  engineers  in  project 
design.  To  .i.clude  updating  of  as-built  drawings  if  performed  by  the 
engineers  or  site  development  personnel. 


INPUTS  FURTHER  DEFINED 
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1.  Labor  manhours-  The  gross  manhours  available  during  the  year 

assuming  an  eight  hour  day,  five  day  week,  work  week.  The  figure  is  j 

2080  hours  a  year  per  person.  Supervisory  personnel  are  included  in  *  1 

the  figure.  i 


2.  Labor  costs-  Figured  using  the  base  shop  rate  times  the  gross 
manhours. 

3.  Years  experience-  The  cumulative  total  years  of  engineering 
experience  of  personnel  in  design.  A  scale  weighting  the  type  of 
experience  could  be  used,  but  would  be  '  ctremely  hard  to  administer. 

a 

•If  all  organizations  use  the  same  criteria,  a  weighted  scale  would  not 
be  necessary. 


! 

4.  Personnel  skill  level  aggregate-  A  numeric  total  of  points  | 

I 

assessed  for  the  personnel  skill  level  according  to  the  following 
point  values: 


GS-7 

3 

AB-A1C 

1 

LT 

4 

GS-9 

4 

SRA-SSGT 

2 

CAPT 

5 

GS-11 

5 

TSG-MSG 

3 

MAJOR 

6 

GS-12 

5.5 

SMSG-CMSG 

4 

GS-13 

6 
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One  additional  point  will  be  added  if  the  individual  is  a  licensed 
professional  engineer.  An  additional  .5  will  be  added  if  the 
individual  possesses  an  advanced  degree. 

5.  Number  of  professional  education  courses  completed-  Courses 
sponsored  by  AFIT,  civilian  university  or  professional  organization. 
Conventions  and  seminars  should  also  be  Included. 

6.  One  over  the  number  of  additional  duties-  This  is  to  account  for 
the  decrease  in  resources  available  to  perform  work  when  personnel  are 
involved  in  additional  duties. 


Appendix  E:  Data 


This  appendix  contains  the  printout  of  the  data  file  used  by  the 
Constrained  Facet  Analysis  model  in  determining  the  efficiencies  for 
the  organizations.  At  the  top  of  the  printout  is  the  title  assigned 
to  the  model  (for  computer  file  identification  purposes)  by  the 
authors.  This  is  followed  by  a  line  that  contains  three  numbers.  The 
first  number  is  the  number  of  organizations  in  the  data  file.  The 
second  number  is  the  number  of  outpuf.3  and  the  last  number  is  the 
number  of  inputs.  The  output  and  input  titles  are  then  printed  out. 
This  order  is  important,  as  it  is  in  the  same  order  that  the  data  is 
listed  within  each  organization.  Each  organization's  data  requires 
four  lines  on  the  printout.  The  organization  numbers  are  printed  on 
the  left  hand  side  of  the  page.  This  data  is  supplied  for  any  future 
researchers  that  may  attempt  to  replicate  the  model  and  result. 
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Appendix  F:  Organizational  Efficiencies 


Organization 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 


Lower  Bound 

1.000 

.79136 

.59427 

.69733 

.88892 

.76336 

.68393 

.57143 

.64095 

.59284 

.65909 

1.000 

.77934 

.99906 

.95138 

.10667 

.96564 

.90801 

.79891 

.71702 

.79896 

.88870 

.00000 

1.000 

.62500 

.59427 

.69733 

.82038 

.76336 

.57895 

.57143 

.44643 

.45455 

.63636 

1.000 

.77929 

.99901 

.67227 

.62500 

.52147 

.0000 

.79888 

.05228 

.0000 

.38867 


Upper  Bound 

1.000 

.79136 

.75203 

.93149 

.88892 

.87833 

.68393 

.75729 

.64095 

.59284 

.96655 

1.000 

.77934 

.99906 

.95138 

.90812 

.96564 

.90801 

.79891 

.71702 

.79896 

.88870 

.51027 

1.000 

.79137 

.75204 

.93150 

.88893 

.87834 

.68394 

.75732 

.59285 

.64096 

.94871 

1.000 

.77929 

.99901 

.95132 

.86267 

.84910 

.90798 

.79888 

.76813 

.79893 

.88867 
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Organization 

Lower  Bound 

Upper  Bound 

46 

.35714 

.47489 

47 

.99893 

.99893 

43 

.90801 

.90801 

49 

.76336 

.87833 

50 

.99906 

.99906 

51 

.30706 

.97160 

52 

.96564 

.96564 

Appendix  G:  Computer  Printout  of  Results 


This  appendix  contains  a  portion  of  the  printout  that  is  received 
as  the  result  of  a  successful  run  of  the  Constrained  Facet  Analysis 
(CFA)  model. ^  By  itself,  this  printout  is  of  marginal  value  to 
the  manager.  The  manager  should  refer  to  the  input  and  output 

reports.  This  printout  permits  the  individual  running  the  model  to 
check  for  errors  and  consistency. 

At  the  top  of  the  printout  is  the  title  of  the  specific  CFA  model 
(i.e.  the  name  of  the  data  file)  being  run.  This  is  followed  by  a 

line  that  contains  three  numbers.  The  first  number  is  the  number  of 

organizations  being  evaluated,  the  second  number  is  the  number  of 

inputs  evaluated,  and  the  third  number  is  the  number  of  inputs 

evaluated.  The  output  and  inputs  titles  are  then  printed  out.  This 

order  is  Important  as  this  is  the  same  order  the  inputs  and  outputs 

are  listed  in  the  organizational  data  (the  organization  number  is 

listed  in  the  far  left  hand  column)  that  follows.  The  first  line  of 
the  organizational  data  is  blank  except  for  two  numbers  on  the  far 
right  hand  side.  The  first  number  is  the  lower  bound  efficiency  and 
the  second  number  is  the  upper  bound  efficiency.  These  values  are 
listed  separately  in  Appendix  F. 


^■Results  from  the  first  32  organizations  are  included.  The 
printouts  from  organizations  33  to  52  were  unsuitable  for 
reproduction. 
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The  output  and  input  data  follows  on  the  next  lines  in  the  same 
order  as  listed  in  the  header.  Each  output  and  input  datura  requires 
three  lines.  The  top  line  repeats  the  actual  observed  value  of  that 
particular  output  or  input,  as  found  in  the  data  file.  The  second 
line  is  the  multiplier  the  model  used  in  determining  the  relative 
importance  of  that  output  or  input.  The  third  line  is  the  value  of 
slack  (output)  or  surplus  (input)  that  is  available  in  that 
organization  for  the  observed  values.  An  efficient  organization  has 
zero  slack  or  surplus. 
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Appendix  H;  Sample  Organizational  Reports 


This  appendix  contains  a  sample  of  the  organizational  reports 
that  may  be  obtained  from  the  computerized  model.  The  first 
organizational  report  presented  tells  the  manager  what  changes  in 
outputs  would  be  necessary  for  that  organization  to  achieve  maximum 
effeciency  with  no  change  in  inputs.  The  outputs  evaluated  are  listed 
along  the  far  left  hand  side  of  the  report.  Next  to  the  outputs  is 
the  column  titled  "Observed  Values’*.  This  column  gives  the  current 
output  values  for  that  organization.  The  next  column  to  the  right, 
titled  "Efficient  Output  Levels,"  shows  the  output  values  that  would 
have  to  be  achieved  by  the  organization  for  it  to  be  100  percent 
efficient.  The  column  on  the  far  right  gives  the  weighting  of  the 

output  to  the  overall  efficiency.  An  organization  desiring  to  improve 

* 

its  efficiency  should  concentrate  on  those  outputs  with  the  highest 
weighting.  At  the  bottom  of  the  outputs  report,  the  inputs  are  listed 
in  the  same  format  except  that  the  second  column  is  titled  "No  I  put 
Changes  Required."  This  portion  of  the  outputs  report  tells  the 
manager  the  relative  importance  the  model  attached  to  each  input. 

Following  the  output  report  is  the  inputs  report.  This  report 
shows  the  managers  what  changes  in  inputs  would  be  necessary  to 
achieve  maximum  efficiency  with  no  corresponding  reduction  in  outputs. 
The  format  is  identical  to  the  previous  report  except  inputs  and 
outputs  are  switched.  Under  the  "outputs"  heading  the  column  is 
titled  "No  Output  Changes  Required"  while  the  inputs  show  the  values 
necessary  to  achieve  100  percent  efficiency. 


BASE  CIVIL  ENGR  DESIGN  SECTION  PROD 
DECISION  MAKING  UNIT  3  GRG  03 


********************** 
*  SUMMARY  OF  RESULTS  * 
********************** 


EFFICIENCY  RANGE  =  59.4  TO  75.2  PERCENT 

MULTIPLIER  FOR  EFFICIENT  OUTPUT  LEVELS  =  1.681 


*********** 
*  OUTPUTS  * 

*********** 


OBSERVED 

VALUES 


EFFICIENT 

OUTPUT 

LEVELS 


PERCENT 
CONTRIBUTION 
TO  EFFICIENCY 


OUTPUT 

01 

5.0 

10.7 

0 

OUTPUT 

02 

6.3 

14.  2 

0 

OUTPUT 

03 

3.4 

5.  7 

.0 

OUTPUT 

05 

122.0 

267.8 

0 

OUTPUT 

08 

3.3 

159.4 

0 

OUTPUT 

09 

0 

„  2 

0 

OUTPUT 

lO 

1.  1 

2.6 

0 

OUTPUT 

1 1 

1.0 

1.7 

.0 

OUTPUT 

13 

e* 

wJ 

5.  4 

0 

OUTPUT  14 

1 50 . 0 

252.  4 

37.7 

OUTPUT 

15 

240.0 

403.9 

21.7 

TOTAL  : 


59 . 4  PERCENT 
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********** 
*  INPUTS  * 
********** 


OBSERVED 

NO  INPUT 

RELATIVE 

VALUES 

CHANGES 

IMPORTANCE 

REQUIRED 

OF  INPUTS 

INPUT  01 

117.0 

100.0 

INPUT  02 

1600.0 

0 

INPUT  03 

390.0 

0 

INPUT  04 

260.0 

0 

TOTAL  :  100.0  PERCENT 
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BASE  CIVIL  ENGF;  DESIGN  SECTION  PROD 
DECISION  MAKING  UNIT  3  ORG  03 


********************** 
*  SUMMARY  OF  RESULTS  * 
*******  tittai.tttttif.ttyt 


EFFICIENCY  RANGE  =  59.4  TO  75.2  PERCENT 

MULTIPLIER  FOR  EFFICIENT  INPUT  LEVELS  =  .594 

*********** 

*  OUTPUTS  * 

*********** 


OBSERVED  NO  OUTPUT 

VALUES  CHANGES 

REQUIRED 


PERCENT 
CONTRIBUTION 
TO  EFFICIENCY 


OUTPUT 

01 

5.0 

0 

OUTPUT 

02 

6.8 

0 

OUTPUT 

03 

3.4 

.0 

OUTPUT 

05 

122.0 

0 

OUTPUT 

oa 

T  *T 
«•••  w 

0 

OUTPUT 

0? 

0 

0 

OUTPUT 

10 

1.  1 

0 

OUTPUT 

11 

1.0 

.  0 

OUTPUT 

13 

2.  5 

0 

OUTPUT 1 4 

150.0 

37.7 

OUTPUT 

15 

240.0 

21 . 7 

TOTAL  :  59.4  PERCENT 
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fc * **** * * *  * 
*  INPUTS  * 
********?.* 


OBSERVED 

EFFICIENT 

REL AT  I VE 

VALUES 

INPUT 

IMPORTANCE 

LEVELS 

OF  INPUTS 

INPUT  01 

117.0 

69.5 

100.0 

INPUT  02 

1600. 0 

701.7 

0 

INPUT  03 

390.0 

192.  1 

0 

INPUT  04 

260.0 

CO 

■ 

n 

o 

0 

TOTAL  :  100.0  PERCENT 
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